Remapping to a Custom Character Hand in Hand Engine (DOC-5020)

This guide will provide an overview of how to remap the output of Hand Engine to match the hands of a custom character so you can stream data from Hand Engine directly to that character.

Requirements

• StretchSense Hand Engine: Already installed (see Hand Engine Installation Guide (DOC-5009)) with IP address binding configured.

• A calibrated glove (see Hand Engine User Guide (DOC-5010))

• An FBX containing your character in a T-Pose in the first frame (see below for details setting up this FBX)

• Hand Engine License Key: Log in to your account page on https://stretchsense.com/my-account/  or email support@stretchsense.com for details

• Operating System: Windows 10 Pro

Please note: The HE asset is set up using the metric system. Please ensure that your target FBX is setup with the same measurement system (working units in centimeters) in the software you are streaming to (for example if using Maya refer here)

Please note: If using our remapping solution in MotionBuilder we recommend using the MB Character Solver. We have found that the HIK solver can cause different rotational offsets than the MB Character Solver. To change to the MB Character Solver you can navigate to the Character Settings in Motionbuilder's Navigator. See the image below:

Setting Up Your Target Character

Setting Up Bind Pose

In order to remap the Hand Engine output to your custom character, you must first create an FBX that has just your target character in it and that character is in a full T-Pose that is saved at the first frame in the FBX. The character must be in a full T-pose, with each hand in an L-Pose or Paddle-Pose. The fingers of the character should align with a major axis of the scene/FBX. Please note this FBX is used only for extracting bone hierarchy and rotation order data from your character. Once the remapping process is complete, you will be able to stream data from Hand Engine to any FBX/scene containing that character.

The key features for each hand are:

1. The palm is facing down and is parallel to the ground plane

2. The fingers are pointing straight out and are parallel both to each other and to the ground plane, and are aligned with a major axis of the scene/FBX e.g. the X-axis in the screenshots below.

3. If setting the T-Pose in an L-Pose, the thumb is pointing straight forward and is both perpendicular to the fingers and parallel to the ground plane, and is aligned with one of the other major axes of the scene/FBX, e.g., the Z-axis in the screenshots below. The thumbnail should be facing towards the body, i.e., the opposite direction to which the fingers are pointing.

Example of correct bind L-Pose

 

Using Remapping in Hand Engine

1. Setup and calibrate your glove as per normal (see Hand Engine User Guide (DOC-5046) for details).

    

2. In the Streaming box, click on the Remap button of the hand you want to retarget to
   
   

   

    

    

3. In the Remap Settings pop-up window, select Scan FBX and select your target character FBX that in the first frame has your character in a full T-pose with the hands either an L-pose or Paddle pose.

   

    

    

    

    

4. If you have already previously saved a binding for this asset you can select this from the Saved bindings for this asset dropdown to reload this mapping, then skip to Step 9.

   

    

    

5. If this is a new binding and you plan to save the binding for later recall, enter a name for the new binding in New binding for this asset, then check the Save to database checkbox at the bottom of the Remap Settings pop up.

   

    

    

   

    

6. Select the root of the hand on the target character from the Root of this hand asset dropdown box. This is the point where the fingers branch out from and the label name may start with hand/wrist/palm etc.

    

   

    

    

7. The bones of the character in the scanned FBX will now populate all children joints from the root node selected in the Bones of this asset column. In the Bones of HE asset column, select the corresponding bone you wish to map to using the dropdown boxes.

   NOTE: In the example below the naming convention between the target character and the HE asset match almost exactly, but there is no requirement for any similarity in the bone names.
   
   

   

    

8. Click Characterize to save the binding and start streaming the remapped hand data from Hand Engine.

   NOTE: Check the Save to Database box to save your binding for this asset in case you want to go back and edit it later

   

    

   You should see within the device panel in the FBX input field the name of the asset you mapped to.
   
   

   

    

    

9. Turn Remapping on (if not HE will stream joint angle data which is conform with the HE hand asset).

   

    

    

10. In case an error happened during step 8 (mapping to the wrong bone or totally forgetting about a bone) you can simply enter the Remap window again, scan the FBX file you would like to remap to, select the saved binding for this asset, change the binding and update this binding profile by hitting the update button. This will save the changes to the database. Click Characterize and you’re ready to go!

     

    

     

11. Once you have completed the remapping steps in Hand Engine, follow the instructions for one of the below guides to stream data to a third party software:

• Streaming from Hand Engine to MotionBuilder (DOC-5014)

• Streaming from Hand Engine to Unity (DOC-5011)

• Streaming from Hand Engine to UE4 (Unreal) (DOC-5012)

• Streaming from Hand Engine to Maya (DOC-5021)

Common Mistakes

Thumb is in a relaxed position. This will cause the thumb to be tucked in too close to the hand

Fingers are curled/in a relaxed position. This will cause the fingers to be over-rotated then creating a fist

 

Fingers and thumb are correctly set, but the hand/arm is rotated relative to the major axis of the global scene. This will cause off-axis rotations of the fingers/thumb when curling them.

 

Hand Engine Bone Naming Convention

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Video - How To Remap

 

 

Revision Summary