Designing Instructional Information

Introducing gestures and interactions

Many users will be unfamiliar with the unique powers and limitations of their hands in XR. Wherever feasible, Ultraleap recommends minimizing the number of tutorials and the amount of help content users are faced with. To achieve this, make more use of object interactions and use abstract gestures sparingly. New concepts should be introduced gradually throughout the experience.

Begin with a positioning tutorial that clarifies the interaction zone, and introduces the first key interaction or gesture that will be required. This might be grabbing an object or forming a pinch gesture.

Thereafter, use an interaction tutorial to teach how to apply this gesture to a UI control or object. For example, pinching to move a slider component.

Further tutorials in the experience can be used to gradually introduce new interactions if required. Wherever possible, reinforce interactions with visual cues and short text prompts.

We also suggest including a help button or similar so users can bring back tutorials whenever they need them.

Positioning tutorials

A positioning tutorial at the start of the experience helps familiarise users with the interaction area and introduce a first interaction or gesture.

Rather than merely depicting the interactions, invite users to learn by performing the key interactions themselves so they can build up muscle memory directly.

An outline depicting the hand / finger arrangement can get users comfortable with the distance and rough position their hands will need to be in order to use the experience, and introduce a gesture. To ensure the user has understood the pose correctly, they must copy and place their hand inside the outline to progress.

Ultraleap hand tracking showing pinch gesture

Use short text prompts to encourage users to place their hands in the outlines. The less there is to read, the lower effort for users.

The outlines visually depict the intended gesture. Text describing gestures can be open to interpretation, so a visual cue is vital at this stage. For example, when prompted to form a pinch, people will do so in a wide variety of shapes and positions.

The tutorial should respond once the user has achieved the correct positioning to confirm success – for example, by turning the outlines green to indicate a positive outcome.

Depth in positioning tutorials

In VR, it can be difficult to determine depth within the interaction zone. For example, when learning a new application, users can have trouble aligning their hands with two–dimensional elements like flat UI panels and hand outlines.

Ultraleap recommend including a three–dimensional reference in tutorials that deal with flat elements like this. For example, the grid in the example above reacts with colour change to indicate the depth of the hands in the interaction zone. This could also be achieved with environmental elements and shadows. Users can then find the correct depth without needing further instruction.

Interaction tutorials

Having started with a positioning tutorial, interaction tutorials should be used to gradually introduce new gestures or interactions. As before, the principle of inviting users to perform interactions themselves is the most reliable way to teach.

Typical places to use an interaction tutorial are:

  • Using a UI control activated by pinching (below)

  • Re–sizing UI panels with two hands

  • Activating a main menu with a gesture

  • Performing locomotion with a gesture

Ultraleap hand tracking showing pinch and drag gesture

This example uses a practice control: A UI component used solely for tutorial purposes.

To focus user attention on the practice control tutorial, there should be minimal distraction in the surrounding environment. The practice control should look and function like the main instances, but can feature short explanatory text and visual guides such as arrows to help with first use.

The tutorial should prompt users to complete the interaction themselves. Upon successful completion, users are returned to the main application and should then be able to successfully use similar controls. Completion of the tutorial should be necessary to progress, to ensure the user understands the interaction.

Decide carefully where to use tutorials so that they don’t interrupt flow. They work best when used at the start of the application, or when introducing new chapters or sections.

Consider making new interactions available to users as each is taught. For example, gradually unlocking them in a menu over time.

Character mimicking

In this alternative tutorial format, a character directly demonstrates new interactions for the user to then perform the action themselves. The character is made using pre–recorded hands.

Ultraleap hand tracking showing pinch and drag gesture with Blocks demo

As humans are extremely familiar with learning by observing others, this is a very grounded tutorial approach.

Characters can be very basic in appearance and still be effective at communicating the correct movements. Text prompts can also be used in addition to the character for extra clarity.

Character mimicking works better for longer sections which involve several steps. (In contrast, interaction tutorials work well to introduce a single interaction in a few moments.) An example would be learning to use a virtually wearable menu, then building on that by learning to create an object, resize, or reposition it.

Character mimicking works best when teaching interactions that are clear from a distance. Interactions using a small UI panel, or making a selection from a virtually wearable menu, are both possible as long as the character is positioned close enough to make the details apparent.

Typical places to use character mimicking are:

  • Manipulating objects in the environment

  • Using a virtually wearable menu

  • Re–sizing UI panels with two hands

Try this example yourself:

Try Blocks Demo

Ultraleap hand tracking showing pinch and drag gesture with Blocks demo

Back to top