Manipulating Virtual Objects

Try these virtual objects in your own projects with Ultraleap’s Unity toolkit:

Unity modules

Virtual Objects – Interaction Engine Example Scene

Familiar interactions from the real world

Virtual objects are at their most intuitive – and useful – when they appear and behave in a manner similar to objects in the physical world. Consistency with a lifetime of physical experience sustains immersion and minimises the learning curve.

This section covers design guidance for object behaviour. For object visual appearance guidance please visit Virtual Objects.

Objects can be picked up, moved, dropped, re–sized, thrown, torn in pieces and much more in whatever way feels most natural to the user. Abstract gestures and poses are not required. Users can interact with virtual objects without needing to master a new precise behaviour.

Going beyond the physical world

Virtual objects can also be manipulated in ways that are not possible with physical objects. Users can experience novel interactions, such as re–sizing objects with their hands or storing them in infinite, virtual pockets. These extra interactions can be learned quickly if they make sense in context and align with the user’s expectation of how the object should behave.

For example, if a user holds an existing virtual block with both hands and moves their hands apart, it should get bigger. Alternatively, the same action can be used to create a new block and place it in the environment.

In the physical world these actions are impossible. But by matching the user’s mental model of how such an interaction would work if it were possible, it’s accepted by the user and their immersion is retained.

Explore object behaviour with:

Blocks Demo

Cat Explorer Demo

One or two hands?

Users will generally use one hand to pick up virtual objects, inspect them and move them around. The exact hand pose used will vary according to the person, and the size and shape of the object.

It often makes sense to use two hands for interactions that involve more direct manipulation of the object, such rotation or resizing. Once learned, two–handed interactions can feel natural and be quick to perform, but new users may benefit from a tutorial.

Interactions for object manipulation

Below are some examples of how virtual objects can be manipulated using Ultraleap’s hand tracking. Note that object interactions and behaviours are not universal, and should vary depending on the type of application. Dropped objects may fall to the floor in a physics–based game, whereas in a virtual meeting room it might be more useful to have objects float in mid–air, ready to be grabbed.

Push / knock object

Users simply push an object as they would in the physical world. The object’s movement and momentum correspond to the hand’s direction and speed.

The object should also react to touch with a noticeable state change – in this case, an illuminated frame.

Try this example yourself with:

Blocks Demo

Pick up and move object

Users select and hold the object with any variation of a pinch or grab. It is intuitive for smaller objects to respond to pinches with two fingers, and larger objects to respond to whole hand grabs.

Users can move their hand in any direction to reposition and open their hand to place the object.

Try this example yourself with:

Pinch Move Demo

Rotate object

Depending on its size, users can pinch or grab the object with both hands, moving them in a circular motion to rotate, much as they would in the real world.

Depending on the application, it is also possible to do this with one hand if objects do not need to be moved. An example would be a rotatable 3D model for use in a collaborative VR tool.

Try this example yourself with:

Pinch Move Demo

Resize object

Users pinch or grab the object with both hands before moving them apart to increase the size, and together to decrease the size. They release the pose to let go of the object and set its new size.

This is an example of an interaction that fits people’s mental model and can be quickly learned, despite not being possible in the physical world.

Try this example yourself with:

Pinch Move Demo

Inspect object

Using a pinch or grab, users select and hold objects with one hand, before using their wrist to turn it around.

Sometimes it’s useful to display a contextual menu, or information relevant to the selected object. It can be convenient to have this appear as a pop–up box when the object is turned to a new orientation.