Hand Engine Remapping Tool Required
      Back to home
      1. StretchSense Knowledge Base
      2. Live Stream Hand Data
      3. Hand Engine Remapping Tool Required

      MotionBuilder - Streaming from Hand Engine

      This document will take you through the steps of connecting to MotionBuilder with a Hand Engine data stream.

      Requirements

      1. Set Up the Remap Tool in Hand Engine

      Remapping is the function of assigning the bones of the hand asset within Hand Engine to drive the correct bones on the asset you are streaming hand data onto in another software. 

      IMPORTANT:

      - This task will need to be completed prior to streaming hands to MotionBuilder

      Set up the T-Pose of your rigs FBX

      In order to remap the Hand Engine output to your custom character, you must first create an FBX that has just your target character in it and that character is in a full T-Pose that is saved at the first frame in the FBX.

      The character must be in a full T-pose, with each hand in an L-Pose. The fingers of the character should align with a major axis of the scene/FBX.

      Please note this FBX is used only for extracting bone hierarchy and rotation order data from your character. Once the remapping process is complete, you will be able to stream data from Hand Engine to any FBX/scene containing that character.

      The key features for each hand pose are:

      1. The palm is facing down and is parallel to the ground plane

      2. The fingers are pointing straight out and are parallel both to each other and to the ground plane, and are aligned with a major axis of the scene/FBX e.g. the X-axis in the screenshots below.

      3. The thumb should be pointing straight ahead, perpendicular to the fingers, and parallel to the ground plane. It should be aligned with one of the other major axes of the scene/FBX, such as the Z-axis in the provided screenshots. Additionally, the thumbnail should be facing towards the body, which means it should be pointing in the opposite direction to which the fingers are pointing.

      NOTE: When using our remapping  in MotionBuilder we recommend using the MB Character Solver. We have found that the HIK solver can cause different rotational offsets than the MB Character Solver. To change to the MB Character Solver you can navigate to the Character Settings in Motionbuilder's Navigator. See the image below:

       

      Character Solver - Mobu

      Example of correct bind L-Pose

       Correct T-Binding Pose

      Common Mistakes

      The thumb is in a relaxed position. This will cause the thumb to be tucked in too close to the hand

      Common Mistakes - Example 1

      Fingers are curled/in a relaxed position. This will cause the fingers to be over-rotated then creating a fist.
      Common Mistakes - Example 2
      Fingers and thumb are correctly set, but the hand/arm is rotated relative to the major axis of the global scene. This will cause off-axis rotations of the fingers/thumb when curling them.
      Common Mistakes - Example 3

      Creating a remap profile in Hand Engine

      1. Open the remapping tool by going to Tools -> Remapping
      2. Select the character FBX you have set up as per the above instructions:
      3. Once the file has been selected, please select the root of the left hand from the left drop-down menu(1), or you can use a previously saved Remap profile(2). 
      4. Hand Engine will detect the remainder of the bones on the hand hierarchy and list them below. 


        This is how the Hand Engine asset is mapped. Please use this as a guide for assigning the bones of your asset to the bones of the hand in Hand Engine:

        hand bone map


      5. The right-hand column contains the hand bones of the asset within Hand Engine. Please select the associated bone to match your character's hand bones from this list. 
      6. Once this has been completed, you can adjust the save name of this mapping at the top right-hand side:
      7. To save this remapping, click on submit at the bottom of this window:


      If you need to make any adjustments to this remapping, you can make the changes and click update at the bottom. It will overwrite the previous saved remap:

      Assigning remap profile to a hand

      1. Please connect gloves and perform a calibration
      2. Click on Remapping Target in the hand setup box, and select your recently added remap. Make sure to add it for each hand.
      3. Once this has been done for each hand, the hands are now ready to stream outside of Hand Engine to your software of choice. 

      NOTE: If you have created or updated a remapping profile, after selecting the remapping target for each hand, please turn the TCP stream OFF and then ON again to make sure the new remapped stream is active. 

      2. Setting up your MotionBuilder Project

      1. Download and Install the StretchSense Hand Engine and MotionBuilder
        NOTE: for this example, we are using version MoBu2019

      2. Launch Hand Engine and calibrate your gloves

      3. In the Streaming box enter your TCP Port number and set the switch to On. Check that the Streaming Status light in the Performer pane has turned green

      4. Next, we need to add the most up-to-date plugin to MotionBuilder. To do this, first ensure MotionBuilder is closed. Then navigate to the Plugins folder you have downloaded from your account at https://stretchsense.com/my-account/software-downloads/, and then navigate to the HE Mobu folder to find the plugin corresponding to your version of MotionBuilder (eg. HandEngineMobu18_150321_1.dll corresponds to MotionBuilder 2018).

        mobu-downloaded-plugins

         

      5. Copy the DLL matching your version of MotionBuilder (i.e. HandEngineMobuXX, where XX refers to the version of Mobu) to the directory:

        <MotionBuilder Install Directory>\Autodesk\MotionBuilder 20XX\bin\x64\plugins\

      6. The plugin is now installed in MotionBuilder, you can import your custom rig into a new or existing scene.

      7. In Resources > Asset Browser > Templates > Devices find the device OR - HandEngine Link and drag this into the scene.

        mobu-HE-device-location

         

      8. In the Navigator > Devices find the new OR - HandEngine Link device and then set the IP Address and TCP Port settings as specified in Hand Engine

        NOTE: The default value for Address is 127.0.0.1 (= localhost), i.e., you are running Hand Engine on the same PC that you are running MoBu. If you have bound Hand Engine to a different IP address (e.g. if you are running Hand Engine and MoBu on different computers on the same LAN), use this IP address instead for Host.

        NOTE: This is the step you need to repeat if you turn Streaming to Off in Hand Engine as previously described.

        mobu-HE-device-plugin

         

      9. Press the online button and you should see the icon turn green and the sample(s)/s value increase signifying data arriving at Mobu.

        mobu-HE-device-plugin-turned-ON

         

      10. From the model binding dropdown select hand_l (note if Right Hand is selected in Hand Engine, the binding option will swap to hand_r).

        mobu-HE-device-plugin-select-model-binding

         

      11. The hand in the scene should now move as you move your hand.

      12. To record animation in MotionBuilder:
        a. First right click on the Action timeline and select Time → Show as timecode,
        b. Ensure Recording is ticked in the Stretchsense HandLink Device,
        c. Click the Record button in the MoBu Transport Controls and create/overwrite a take.
        d. Click the Play button in the MoBu Transport Controls to begin recording.

        NOTE: Starting to record a take in MoBu will trigger Hand Engine to begin recording raw data simultaneously. The take name from MoBu will be sent to Hand Engine and the raw data file recorded by Hand Engine will be recorded in a new folder that follows the <take name>_<date>_<time> format.

        NOTE: To replay the take in MoBu, ensure Live is unticked, right-click on the Action timeline and select Time → Frame Take, then click Play.