This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(294.40000000000003, 89%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPY%3C/text%3E%3C/svg%3E)
I’m developing an application for Hololens 2 using Unity 2021.3 LTS and Visual Studio 2022. I want to implement a feature where the user can gaze at a virtual object, select it by making a pinch gesture (thumb and index finger closing in air), move the object by moving their eyes or head (while keeping the pinch gesture), and finally place the object at a desired location by releasing the pinch gesture (thumb and index finger opening).
I’m having trouble figuring out how to implement this. I’ve tried using the MRTK toolkit but I’m not sure how to correctly use the gaze and gesture inputs to move the object. Any help or guidance would be greatly appreciated. Please let me know if you need more information about my setup or the specific problem I’m facing. Thank you!
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(198.4, 72%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EYL%3C/text%3E%3C/svg%3E)
Hello, Welcome to Microsoft Q&A,
To implement your requirements, you need to use eye tracking. Please refer first to the following link: Eye Tracking Basic Setup - MRTK 2 | Microsoft Learn.
You can use EyeTrackingTarget to implement your requirements for using eye gaze and hand gestures to move objects. For more details, please refer to this example - Eye tracking examples overview - MRTK 2 | Microsoft Learn. To acquire this example, please refer to Example scenes - MRTK 2 | Microsoft Learn.
According to Eyes and hands - MRTK 2 | Microsoft Learn, you may need to disable hand rays when you are using a combination of eye gaze and hand gestures as your interaction scheme. Due to potential version differences in the MRTK package, you may not see "DefaultItControllerPointer" in the pointer settings. So, we recommend that you use the input profile EyeTrackingDemoPointerProfile in the example scene mentioned above as a reference.
If the response is helpful, please click "Accept Answer" and upvote it.
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.
Dear Ying Li,
Thank you for your guidance. I have reviewed the documents you suggested, but I found that they do not fully meet our needs. The document (Eyes and hands - MRTK 2 | Microsoft Learn) describes "Positioning: Fluently move a hologram across your scene by simply looking at it, pinching your index finger and thumb together to grab it and then move it around using your hand." However, our requirement is to gaze at a virtual object and then select it by "pinching", continue to move the object by gazing (not by hand), and finally gaze at a location and place the object by "pinching".
I appreciate your help! Do you have any other suggestions or methods that might be more suitable for our specific needs?
Best regards.
Sorry for the error in the previous edit, I’ve updated the first link, please check it again.
Regarding the eye-tracking example I previously recommended, I believe it can meet your requirements for “gaze at a virtual object and then select it by ‘pinching’, continue to move the object by gazing (not by hand), and finally gaze at a location and place the object by ‘pinching’”. Please note that the eye tracking example includes several sample scenes, I recommend you try this sample scene first, which is your target：
This answer has been deleted due to a violation of our Code of Conduct. The answer was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Comments have been turned off. Learn more