How To Get Your Hand Assets Ready For Unity

We recently released our new Hands Module, part of our Unity Modules package.

We’re very keen to support you in making your own hand assets, so we’ve put together this guide to the principles for rigging your own creations.

Sketch of robot hands

Use our rigging guide to get your sketches off the page and attached to our tracking data

If you’ve got some experience with modelling and you want to get a bit more technical, you’re in the right place.

This page will:

  • Show you how to rig your own hand assets so they work with our tracking data.

  • Detail how to use our new Hand Binder pipeline, as well as some of what happens under the hood.

  • Break down the 3D rigging process and walk you through how to connect to the 27 possible data points of the hand.

  • Cover some best practices for both building hand assets from scratch or using hand assets from a 3D asset store.

Sketch of cartoonified hands

Your hand doesn’t need to have the typical number of fingers and thumbs…or even be a hand. You rig your creation the way you want, and you decide how it works with our tracking data

It doesn’t matter which modelling package you’re using. We use Autodesk Maya, so some of the screenshots we use show the Autodesk Maya interface. The rigging principles are the same across different modelling environments even though the terminology may differ.

To make things nice and clear, we’re going to demonstrate on a 3D model of a skeleton hand.

Skeleton hand model

Connecting to the 27 Data Points

We can use the positional and rotational data from the following points to connect to our rigged 3D Hand.

  • Wrist

    • Thumb_Meta

      • Thumb_Proximal

        • Thumb_Intermediate

          • Thumb Distal

            • Thumb_Tip

  • Index_Meta

    • Index_Proximal

      • Index_Intermediate

        • Index_Distal

          • Index_Tip

  • Middle_Meta

    • Middle_Proximal

      • Middle_Intermediate

        • Middle_Distal

          • Middle_Tip

  • Ring_Meta

    • Ring_Proximal

      • Ring_Intermediate

        • Ring_Distal

          • Ring_Tip

  • Pinky_Meta

    • Pinky_Proximal

      • Pinky_Intermediate

        • Pinky_Distal

          • Pinky_Tip

  • Elbow

We cross out Thumb_Meta because its data is identical to the Thumb_Proximal. You can include a Thumb_Meta if you wish, but your 3D Hand will give you the same results.

You don’t need to use all 27 points. Maybe your hand has fewer than the usual number of fingers, or maybe your ‘hand’ isn’t a hand. If your hand is a clamp or a claw you won’t need 1:1 mapping. Within your game engine you can connect to just 1, or all of these joints. Whatever works.

Joint placement

Skeleton hand model

Now that we know what data we can use, it’s time to decide where we should position each bone/pivot onto our 3D model.

Each joint will contain a gizmo that points towards the next bone. Using this you can line up the joints so they sit in the centre of the mesh. Change the viewport to top down view and enable Projected Centering. Projected Centering estimates the centre of a mesh volume to help you position your joints.

Each time you make a new joint it will assign the previous joint you made as the parent. This relationship is visually identifiable – as you can see below, each metacarpal joint (connecting the palm to the fingers) is attached to the single parent wrist bone.

Skeleton hand model

It’s important to think carefully about where you want the bone to pivot around. In this example you can see each new bone is at the end of the previous bone.

Joint Orientation

So now the bones are in the correct place, the next step is to make sure they are all pointing the right way. This process, called orienting joints, makes sure that the rotation axis is set up for how you’d like the geometry to rotate around the joint.

Make sure each joint is orientated so that :

  • The Primary/Forward axis points down the bone towards the next bone of the hand.

  • The Secondary/Up axis points out the top of the hand.

  • The final axis will point out of the finger either to the left or the right of the hand model.

The tracking data has been defined so that:

  • Z - Points down the bone.

  • Y - Points up out of the bone.

  • X - Points to the left of the bone.

To achieve this in Maya, you can set the Orient Joint Option to these values. If you turn on Orient Children of Selected Joints it will also automatically orient any children of the bone you have selected.

Control panel

It’s important to inspect the geometry of your model as you rotate joints. Think about how you want the geometry to move once it is attached to that joint. Make sure each axis lines up with how you would expect it to rotate around.

We’ve found that Maya has really helpful little graphics that show you the rotation axis of each joint. Below is a screenshot of the Local Rotation Axis of the bones on the index finger.

Skeleton hand model

As we rotate the joint on each axis, we can see how the bones will rotate when attached to the tracking data.

In comparison, you can see how this joint with an incorrect rotation axis will behave.

Helpful tips for Maya users:

  • Once you have orientated all the joints, make sure you freeze the joint transformations. Doing this sets the current rotation of the selected joint as zero. This means if you accidentally rotate the joint, you can restore it back to the rotation you specified after following the steps above.

  • Set the preferred angle. This will store the orientation of the joints for you. You can then hold down right click over the topmost joint and select Assume Preferred Angle to reset the entire joint structure back to how you have just positioned and orientated it.

Weight Painting/Skinning

Now you’ve got the joints positioned and orientated onto your model, the next step is attaching the geometry to these joints. To do this, you are going to follow the process of weight painting – sometimes referred to as skinning. Weight painting is the process of assigning a weight to each vertex of a model. This weight is between 0–1 and is the amount each joint will affect that vertex.

Note – After binding the geometry to the joints you should not be changing the geometry in any way.

To bind the geometry, select the top-most joint and the geometry you wish to bind. Then select Bind Skin from the Skin Options menu.

When you bind the joints for the first time, you might notice that the geometry is not connected very well. To fix this, inspect the weight paints and see which joints are connected to which vertices of the model.

In the picture below we can see the wrist bone is currently affecting the radius bone of the model. While this may look correct, it’s important to note that joints work down the bone. In other words, the wrist joint should only be affecting geometry between itself and the next bone in the chain.

Skeleton hand model

You can clearly see the impact of getting this wrong in the clip below. If the wrist joint had the weighting shown above, the result would look something like this:

To fix this, we need to select the joint before the wrist, then give all the vertices of the Radius and Ulna a weighting of 1. Notice in the video below that when we rotate the wrist this time, the geometry stays in the correct place.

Once you’ve assigned the correct weights to each joint, test by moving the joints around. Pay attention to how the joints and geometry move together.

Joint naming

Next, name your joints. Simply select a joint in the Outliner and type the name.

A few examples of naming conventions are shown below but for best results follow the naming convention outlined in Connecting to the 27 Data Points.

Joints that are named joint1, joint2 …etc. will not work with our auto rigging scripts and you will need to manually assign each bone.

Joint naming panel

Let’s Have A Show Of Hands Please

And that’s it! We’d love to see your creations. Weird, wonderful, hyperrealistic, monstrous or otherwise. Tag us in your work (@ultraleapdevs) and we’ll share on our platforms.

Want to stay in the loop on the latest Ultraleap updates? Sign up to our newsletter here.

We welcome feedback on our products and services. Check out our support centre or contact us.