Hand Engine 2.0 User Guide

This guide will provide an overview of how to use Hand Engine 2.0 to connect to the MoCap Pro Gloves and create hand animation.

Requirements:

  • StretchSense Hand Engine 2.0 or above: Already installed (see StretchSense Hand Engine Installation Guide (DOC-5009)) with IP address binding configured if required.

  • StretchSense Hand Engine version 2.0.
  • Hand Engine License Key (viewable on your account page on the Downloads page)

  • Minimum System Requirements:

    • Windows 10 Operating System
    • CPU 2.5GHZ
    • GPU Dedicated Graphics
    • RAM 8GB

Warning: Hand Engine stability is not guaranteed with Windows 11. Please note that you may encounter some software issues if using Windows 11.


Contents:

  1. Setting up your MoCap Pro Glove Hardware
  2. License Key Activation
  3. Creating and Staging a Performer.
  4. Connecting your gloves to Hand Engine.
  5. Import/Export Calibration
  6. Express Calibration
  7. Advanced Calibration
  8. Training the Hand
  9. Setting up Timecode (Vicon, Optitrack)
  10. Setting up Remote Trigger (UDP)
  11. Recording
  12. Recording File Information
  13. Playing back raw recordings.
  14. Remapping to a custom character
  15. Streaming to other software
  16. Batch Processing data recorded to SD Card

 1. Setting up your MoCap Pro Glove Hardware

USB Bluetooth Dongle

Each MoCap Pro glove uses one USB Bluetooth Dongle. Out of the box, these are already paired to each glove. 

  1. Please connect the USB Bluetooth Dongle to any USB Type-A port on your PC (Please see Tips below for more information)
  2. The USB Bluetooth Dongle will have a solid green LED showing. This means that it is waiting for a connection to its paired MoCap Pro Glove.
  3. Power on the MoCap Pro Glove by pressing the power button on top of the glove once.
  4. A slow flashing blue LED on the glove means that it is not currently connected to a USB Bluetooth Dongle (One second on, one second off)
  5. The glove will automatically connect to its paired dongle. Once connected, the glove LED will change to a flashing blue LED (One quick flash every second) and the USB Bluetooth Dongle will have a solid blue LED.
  6. Windows will detect the USB Bluetooth Dongle and automatically apply a COM port number to that dongle.  (You may check these in the Windows Device Manager)
  7. You are now ready to create and stage a performer and connect to Hand Engine.

Tip: Using a USB Hub

You may use a USB Hub to connect your USB Bluetooth Dongle.

Please make sure that the USB Hub you are using has its own independent power supply. This is very important if the hub is located at the end of a USB extension cable. 

 

 

Tip: For the best Bluetooth signal

The antenna on the USB dongle is designed to radiate perpendicular to the dongle. 

To ensure the best signal performance:

  • Position the dongle off the floor. Placing the dongle near the floor degrades the signal quality.
  • Position the dongle vertically. This ensures the antenna is in the correct orientation for best performance. 
  • Use a powered USB hub if required.
  • Avoid plugging dongles in behind a desktop PC. The proximity to other electronics, power cables, and metal objects will degrade the signal. 

 

 

 

Wired USB Connection

You may connect your MoCap pro gloves to your PC using a Micro USB cable in a wired configuration. This may be helpful when doing desktop capture sessions and a wireless connection is not required.

  1. Please connect your glove to your PC using a Micro USB cable. If you are using a Fidelity glove, skip to step 3.
  2. On the side of the SuperSplay glove electronics module, please change the switch from C (Charge) to D (Data). 
  3. Power on the glove by pressing the power button once
  4. One powered on, please press the power button on the glove twice (quickly)
  5. The LED on the glove will change from blue to a quick flashing green color (One flash every second)
  6. You are now ready to create and stage a performer and connect to Hand Engine.

Pairing USB Bluetooth Dongle

Out of the box, your glove should already be paired to its associated USB Bluetooth Dongle.

If your glove is no longer paired to the USB Bluetooth Dongle, or you have received a replacement dongle, please follow these instructions to re-pair.

  1. Please turn all gloves off in the vicinity.
  2. Make sure the USB Bluetooth Dongle is connected and is showing a solid green LED.
  3. Press and hold the button on the top for approximately 10 seconds while the dongle is plugged in.
  4. The LED on the USB Bluetooth Dongle will turn solid red.
  5. Power on the glove and place it within 10cm of the USB Bluetooth Dongle.
  6. The dongle LED will change to solid green, and then to solid blue once the glove has connected.


         
    2. License Key Activation

    The first time Hand Engine is launched it will open on the activation screen requesting a license key. The key can be obtained from your account on StretchSense Downloads

    If you do not already have an account you can sign up here  Create an Account - StretchSense.

    Contact our Support team at support@stretchsense.com if you have any issues or if your license key is not in your account.

    Automatic Activation:

    Copy and paste your license key into the required box, then click on Activate.

    If Hand Engine fails to activate, follow the information from the error pop-up. Contact our Support team at support@stretchsense.com if you are unable to activate your license.

    Manual Activation:

    Click on the manual activation option, paste your license key, and click on 'Save Activation Request'. 

    Hand Engine will generate an .XML file, which will need to be sent to our Support team at support@stretchsense.com, then we will send you back an Activation Response .XML file.

    Go back to Hand Engine and click on 'Set Activation Response' to load the file provided by our Support team, then click on Activate

    Floating License Activation:

    With the Floating License, you must first install the Floating Server on a computer on your network. Once it has been installed and activated, the server has a “pool” of “license leases” (i.e., the maximum number of instances of your app that can run at any given time Hand Engine can only run 1 instance at a time). Then you can install Hand Engine on as many computers as you want.

    This also works over a VPN, so if you have other locations that require access to the Hand Engine license, you can point them towards the floating license server as long as they are on the same VPN network.

    When you launch Hand Engine, the following happens:

    1. Hand Engine tries to connect to the Floating Server installed on the network. If Hand Engine can successfully connect to the Floating Server, it requests a “license lease” from the Floating Server instance. One of two things will happen:

      a) If there's 1 “license lease” left in the “pool” then Hand Engine will get that “license lease”.

      b)If there are 0 “license leases” left in the “pool” then Hand Engine will not be able to start.
       
    2. When you close Hand Engine, it releases that “license lease” back into the “pool” so that it can be used by other users in that network.

    In Hand Engine, paste your license key into the appropriate box and click on 'Install floating license server here'

     A command prompt window will open and Windows will request for admin access twice during the installation.

    Once the process is complete click on Activate to finish the activation process and open Hand Engine on the server PC.

    When you close Hand Engine, the license lease will be released back to the server and available for another PC to open Hand Engine.

    The floating license server is now installed on this PC.

     

    On any other PC that wishes to activate using the floating license:

    1. Select floating license at the license activation page of Hand Engine.
    2. Enter the license key.
    3. Enter the IP address of the PC running the floating license server.
    4. Leave the port as default (13).
    5. Click on Activate.

    3. Performer Setup

    Create a performer:

    Hand Engine has the ability to create and store calibration for multiple performers. Before connecting gloves, you will need to create and stage a new performer.

    1. At the bottom left-hand side of Hand Engine, please enter a new performer name and click on Add.
    2. The performer will be created and automatically staged. 

    Stage an existing performer

    If a performer is already staged, the check box next to that performer will be ticked. To stage a performer, please select the check box next to that performer.

     

    Unstage a Performer

    To unstage a performer, please de-select the check box next to that performer. You can select this box to bring the performer back to the stage.

     

     


    4. Connecting your Gloves to Hand Engine

    There are three methods for connecting your MoCap Pro gloves to Hand Engine:

    Pre-requisite: Please stage a performer first to allow gloves to be connected/associated with that performer.

    To connect to your gloves, please stage a performer, and next to the associated hand (Left or Right), click on the drop-down box to select the connection mode:

     

    USB Bluetooth Dongle Connection

    Please select the associated com port for your USB Bluetooth dongle and Hand Engine will connect the hand to that dongle. (Make sure your glove is turned on for data to stream)

    Wired USB Connection

    Please follow instructions for connecting gloves in a wired configuration.

    1. Please make sure the switch on your glove electronics module is set to "D" (Data)
    2. Plug the micro USB cable into your glove, and into your PC. 
    3. Power the glove on, and then press the power button once momentarily. The LED light on the glove will change to a quick flashing green.
    4. In the Hand Engine connection dropdown, select the correct USB connection to connect the glove:

    WiFi Bridge App Connection

    You can download the StretchSense Wifi Bridge application here:

    https://play.google.com/store/apps/details?id=com.stretchsense.smartapp

    Pair your gloves to the app and take note of the port numbers in use. The port numbers start from 9000 and increments for each additional glove connected. (Glove 1: Port 9000, Glove 2: Port 9001)

    Please select Add network port and enter the port number the associated glove is set to on the WiFi bridge application. 

    Note: The default port starts from 9000 for the first glove and increments (i.e. Glove 1: 9000, Glove 2: 9001)

    Once you have entered the correct port number, please click on Submit to connect.


     

    5. Exporting and Importing Calibration

    Hand Engine 2.0 has the ability to export and import calibrations for your performers. This can be very useful if moving between machines, or if you wish to keep a backup of your calibration.

    Calibrations are saved as a .json file.

    Exporting a Calibration:

    1. Make sure a glove is connected and calibration has been completed (See section 5  and section 6)
    2. Focus Performer and click on Advanced Calibration
    3. Within the Hand Training tab, please click Export Calibration button 
    4. Give your calibration file a name and save it somewhere convenient.

    Importing a Calibration

    1. Make sure a glove is connected and calibration has been completed (See section 5  and section 6)
    2. Focus Performer and click on Advanced Calibration
    3. Within the Hand Training tab, please click Import Calibration button 
    4. Navigate to where your calibration files are saved and open the calibration file you require. 
    5. Once the calibration has loaded all the poses, please select the poses you wish to use for calibration (Blend or Pose), or turn "Include Express Calibration" and click the Train button.

    Calibration Files containing Express Calibration Only:

    When importing a calibration that only contains Express Calibration (and no captured poses), you will need to capture a single pose (any pose) before you can click the Train button. You can then deselect this pose from the Blend column and click Train a second time to return to your imported Express Calibration only

     




    6. Express Calibration Capture

     

    Please make sure you have staged a performer and connected your gloves to Hand Engine prior to performing a calibration.

    1. Make sure your glove is connected, the performer staged and data is coming through.

    2. Click on the Calibrate Express button
    3. Please perform hand motions as per the below video
      HubSpot Video
    4. Once you have completed the Express Calibration, you can start recording or streaming right away! or try the Advanced Calibration method.

    Tip: Changing Express Calibration Duration

    While you will achieve amazing results with a 15-second calibration, you may wish to have a little more time to complete the process.

    You can change the Express Calibration Duration in the Settings menu as shown below:


    7. Advanced Calibration - Pose Capture

    1. Click on Advanced Calibration next to the performer's hand you wish to calibrate.

       

    2. The Advanced Calibration window will pop up.

    3. Next, Select the pose library that contains the poses you wish to capture for your calibration

       
    4. Select the pose you wish to start with. The viewport hand as selected will change to the selected pose
       
    5. Have the performer replicate their hand exactly as seen on the viewport hand shown to the side of the Advanced Calibration window. 
    6. Have the performer rotate their wrist slightly whilst holding their hand in the correct pose
    7. While rotating the wrist and holding the pose, please click on the "Capture Hand" button.
    8. The capture takes a couple of seconds, please maintain pose and wrist rotation until the button turns orange again. 
    9. If you have "Auto Advance Pose" Selected, the viewport hand will move to the next pose in the library automatically.
    10. Repeat steps 6 to 9 for each of the poses in the library as required.

    If you have captured a pose incorrectly and wish to delete that pose. Please click on the Hand Training tab and click on the delete button next to the pose you wish to delete. 

    Tip: You can click and drag the viewport to rotate the hand. This makes getting a better view of the pose easier so you can replicate it exactly.

    IMPORTANT! For best results, have the performer initially copy the reference pose with their hand with the wrist straight.

    It is very important to make sure all of the joints on the performer’s hand match the reference hand pose from the pose library, if the performer cannot achieve the pose, they shouldn't train to that pose than to train it incorrectly.

    Furthermore, it is highly beneficial to ensure that if a particular joint is in the same position in more than one pose, that for each of these poses the corresponding joint on the performer’s h vand is also in the same position.

    This provides higher quality data to the hand solver that results in optimal calibration performance. 

    Once the performer is set in the hand pose, ask them to hold the hand pose steady and hold their forearm steady while gently rotating at their wrist (e.g. like the wrist would move when using a skipping rope), and to keep rotating their wrist in this circular motion until the capture is complete.

    Start the capture process by clicking the Capture button. The capture process takes <2 seconds for each captured pose.




    8. Training the Hand

     
    Hand Engine 2.0  has various options for training the output hand, making it an extremely flexible tool to achieve the results you require.
    To access these options, please enter the Advanced Calibration section of Hand Engine 2.0 and click on the Hand Training tab at the top. 
    Here you will find all the settings required to make adjustments to your calibration.

    Express Mode


    If you have performed an express calibration. Hand Engine 2.0 will default the hand output to using the express mode.

    If you have used an Advanced Calibration training method such as Blend, Pose, or Hybrid and wish to go back to the Express output mode:
    1. Open the advanced Calibration Section
    2. Select the Hand Training tab
    3. Deselect any poses under the Blend or Pose columns
    4. Select the "Include Express Calibration" toggle
    Click on "Train" to set the output to Express Mode

    This switch will turn on or off any express calibration that you have done. If you wish to use a blend calibration (Where Hand Engine will only use captured poses as its calibration source), please turn this switch off and see below for how to use the blend mode. 

    Blend Pose Mode

    If Express mode is turned off, Selecting poses in the blend column will include these as part of the base calibration. Please add the poses you see fit to achieve the calibration quality you require. 
    1. Open the advanced Calibration Section
    2. Select the Hand Training tab
    3. Ensure you have performed an Advanced Calibration and captured a set of poses.
    4. Turn "Use Express Mode" off
    5. Select the poses you have captured under the Blend column. 
    6. Click on "Train" to set the output to use your captured poses as a Blend mode calibration.

    Key Pose Mode

    By selecting one of the checkboxes under the Pose column, Hand Engine will be able to Key (or lock) to that pose.
    If there are no Blend poses selected, and "Use Express Mode" is turned off, Hand Engine will only output the key poses you have selected under the pose column.
    1. Open the advanced Calibration Section
    2. Select the Hand Training tab
    3. Ensure you have performed an Advanced Calibration and captured a set of poses.
    4. Turn "Use Express Mode" off
    5. Deselect any poses that may be ticked under the Blend column.
    6. Select the poses you wish to pose mode under the pose column
    7. Click on "Train" to set the output to use your captured poses as a Blend mode calibration.

    Hybrid Calibration Options

     Tuned Express

    Tuned Express is the combination of Express calibration with some blend poses to improve the overall accuracy of the express calibration.

    1. Perform an Express Calibration
    2. Open the Advanced Calibration section
    3. Capture poses you wish to use to improve upon the express calibration you have done

    Before selecting poses, assess where the calibration needs improvement. A great way to do this is to check poses such as Paddle, Fist, and Thumbs up, and see if the accuracy of any of the fingers needs improving in these areas. 

    1. Select the Hand Training Tab
    2. Make sure that "Include Express Calibration" is turned on
    3. Select the poses you wish to use for the tuned express under the Blend Pose column

    4. Click on Train and assess your calibration. 

    Express with Key Pose

    Express with Key Pose is the combination of express mode calibration and selecting key poses.

    1. Open the advanced Calibration Section
    2. Select the Hand Training tab
    3. Ensure you have performed an Advanced Calibration and captured a set of poses.
    4. Turn "Include Express Calibration" on
    5. Deselect any poses that may be ticked under the Blend column.
    6. Select the poses you wish to key under the pose column
       

       Important! - Selecting Key Poses for Hybrid Modes:

      Please select a minimum of two poses for this to operate correctly. The two poses need to be at opposite extremes of joint angle. You are able to set the key confidence of the opposing pose extremely low if you do not wish Hand Engine to key to that pose. 

      Example: I wish to key to the Fist pose, I will need to select the paddle pose and I will set the key confidence of Paddle to 5, Fist I will set key confidence to 15 as I wish to key into this pose.

        

    7. Click on "Train" 

    8. Test the sensitivity of the key poses, and adjust the Key Confidence higher or lower depending on how strongly you wish to lock into these poses. 
    9. If you have changed any key poses or key confidence, please click the Train button again to update the calibration. 
    10. All other hand motions besides the key poses will be based on the express calibration.

    Blend Pose with Key Pose

    Blend Pose with Key Pose is the combination of a blend base calibration and selecting key poses. 

    1. Open the advanced Calibration Section
    2. Select the Hand Training tab
    3. Ensure you have performed an Advanced Calibration and captured a set of poses.
    4. Turn "Use Express Mode" off
    5. Select the poses you wish to use for the blend calibration under the blend column.
    6. Select the poses you wish to key under the pose column
       

       Important! - Selecting Key Poses for Hybrid Modes:

      Please select a minimum of two poses for this to operate correctly. The two poses need to be at opposite extremes of joint angle. You are able to set the key confidence of the opposing pose extremely low if you do not wish Hand Engine to key to that pose. 

      Example: I wish to key to the Fist pose, I will need to select the paddle pose and I will set the key confidence of Paddle to 5, Fist I will set key confidence to 15 as I wish to key into this pose.  

    7. Click on "Train" 
    8. Test the sensitivity of the key poses, and adjust the Key Confidence higher or lower depending on how strongly you wish to lock into these poses. 
    9. If you have changed any key poses or key confidence, please click the Train button again to update the calibration. 
    10. All other motions besides the key poses will be based on the blend calibration. 

    Tuned Express with Key Pose

    Tuned Express with Key Pose brings together the power of Tuned Express mode with key poses that will snap into place when the hand is in that pose. 

    1. Perform an Express Calibration
    2. Open the Advanced Calibration Section
    3. Capture poses you wish to use for tuned express and the poses you wish to key.
    4. Click on the Hand Training tab
    5. Make sure that "Include Express Calibration" is switched on
    6. Select the poses in the Blend column you wish to use to improve the express calibration (Tuned Express)
    7. Select the poses in the Key Pose column you wish to have the hand snap into.
    8. Adjust the key confidence as required.

     

    9. Setting up Timecode

    Your timecode source can be set from the Timecode section at the bottom right-hand side of Hand Engine 2.0

    You have three options for timecode source:

    • System Clock
    • Vicon 
    • Optitrack

    You are able to check the source timecode within the recording section at the bottom center of Hand Engine. 

    Caution! 

     

    Prior to recording any take, please ensure the timecode shown here is matching your source timecode. 

    System Clock

    1. Please select System Clock from the dropdown menu
    2. Hand Engine will now use the local PC system time as its timecode source

    Vicon Timecode

    1. Make sure you have Shogun running with an active TCP DataStream
    2. Select Vicon from the Timecode Source drop-down menu
    3. Enter the IP Address and Port number of the data stream from Vicon
      1. The default port for Shogun and Blade is 801. If you have changed this in Vicon, please ensure the matching port number is entered in Hand Engine
      2. If Vicon is running on another PC on your network, please enter the IP address of the PC running Vicon into the IP Address:Port field.



    4. Click on Connect
    5. Check the timecode source under the Start Recording button in Hand Engine 2.0 and ensure that it is synced to the timecode coming from Vicon.

    Vicon Default Port: 801

    OptiTrack Timecode

    1. In Motive, the Data Streaming pane can be accessed under the View tab or by clicking the Toolbar Stream 20.png icon on the main toolbar.
      1. Motive 3.0 is located in the Edit -> Settings option. 
    2. Within the streaming pane, make sure you have selected "Broadcast Frame Data" and please take note of the transmission type as this will need to be selected correctly within Hand Engine
      1. If streaming to Hand Engine on another PC (Network), please set the local interface to the IP address of the PC running Motive. (Run ipconfig in a command prompt window to find this)
      2. If streaming to Hand Engine on the same PC (Localhost), set local interface to "Loopback"

        File:DataStreaming Pane 21.png
    3. Within the Hand Engine 2.0 Timecode section, please select OptiTrack from the source dropdown menu
    4. Within the IP Address:Port field, please enter the IP address of the PC running Motive and the port number. The default for Motive is 1511.
    5. Within the Local Interface IP, please enter as follows
      1. If connecting to motive on another PC (Over network), enter the IP address of the PC running Hand Engine. (You can find this by running ipconfig within a command prompt window)
      2. If connecting to motive on the same PC (Localhost), Please enter localhost IP: 127.0.0.1
    6. Please make sure you have selected the correct option for the Cast field (Unicast or Multicast) as set within the motive streaming NatNet settings.
    7. Click on Connect
    8. Check the timecode source under the Start Recording button in Hand Engine 2.0 and ensure that it is synced to the timecode coming from Motive.

    Motive Default Port: 1511

     

    10. Setting up Remote Trigger

    You have the option to set up a remote recording trigger from the following systems:

    • Vicon
    • OptiTrack
    • Xsens
    • Qualisys

    Setting up a remote UDP trigger will instruct Hand Engine to start and stop recording, as well as set the take name for the recordings being performed. 

    Hand Engine Settings

    1. Please open the Settings window (Windows -> Settings) from the top left menu.
    2. Within the settings window, you will find the Remote Triggering Settings.
    3. Please select your Remote Triggering Source (Vicon, OptiTrack, Xsens or Qualisys)
    4. The default ports have already been set up, if your remote trigger port is different, please adjust.
    5. Turn on the Remote Triggering Enable switch.

    6. Click on Update Settings.

    Vicon Settings

    1. Within the capture settings in Shogun, please select the broadcast capture check box. 
    2. Take note of the port number. By default, this should be port 30.
    3. Within Hand Engine, ensure you have selected Vicon as the remote triggering source within the settings window. 

    OptiTrack Settings

    1. In Motive, Please open the Data Streaming pane. This can be accessed under the View tab or by clicking Toolbar Stream 20.png icon on the main toolbar.
    2. Switch the remote trigger option on. Note: Local Interface needs to be set to the IP of the PC running Motive if you are triggering Hand Engine on another PC on the network. Please set this to loopback if you are running Hand Engine on the same PC as Motive. 
    3. Within Hand Engine, ensure you have selected OptiTrack as the remote triggering source within the settings window.

    Xsens MVN

    1. Within MVN, Please head to Options -> Network Streamer
    2. Add a stream
    3. In the host section, enter the IP address of the PC using Hand Engine 2.0
    4. Take note of the port number. The default port for Xsens is 9763. If this is not entered under the port column, please enter port 9763 here.
    5. Select only "Network Sync XML" as per the below image

    6. Within Hand Engine, ensure you have selected Xsens as the remote triggering source within the settings window and check that the Port number is correct. 


    11. Recording in Hand Engine 2.0

    The recording section of Hand Engine can be found at the bottom center of the window.

    Setting Recording Location:

    Please click on the Set Recording Location button to set the folder you wish to save your recordings

    Entering a Take Name

    Next to the Set Recording Location button, you will find the field to enter your take name:

    Browse Recordings

    Next to the take name field, you will find a button that will open your recording location folder so you are able to browse and review your recordings:

    SD Card Recording

    If you wish to also record on the SD card in your gloves, please set the SD Card option to "Triggered by Record"

    Setting this option to "Off" will not instruct the gloves to start recording to the SD card. 

    See here for more information on reviewing and processing data recorded to an SD card. 

    Starting and Stopping Recording

    To start recording hand animation, please click on the Start Recording button

    To stop recording, please click the same button. 

    Remote Trigger:

    Recording can be started and stopped using a remote trigger from other systems such as Vicon, OptiTrack, Vicon and Qualisys.

    Please see here for how to set up remote triggering.

     


    12. Recorded File Information

    Recordings done within Hand Engine will produce the following files within the recording location folder you have selected:

    1. Calibration .json
      1. File Naming Convention: Cal_<Performer number><Hand Side>_<Performer Name>.json
      2. A calibration file is created for each hand (Left and Right) of the performer
    2. Full Body FBX
      1. File Naming Convention: <Performer number>Fullbody_<Performer Name>.FBX.
      2. This .FBX contains the animation of both hands attached to a full skeleton.
    3. Hands-Only FBX
      1. File Naming Convention: <Performer number><Hand Side>_<Performer Name>.FBX
      2. This .FBX contains the animation of one hand in a hand mesh.
      1. Raw Data CSV
        1. File Naming Convention: <Performer number><Hand Side>_<Performer Name>.csv
        2. A raw data CSV is created for each hand (Left and Right) of the performer.
        3. File Contains:
          1. Raw capacitance data of sensors (16 columns)

          2. Raw IMU data (if applicable) (10 columns)

          3. Timecode (device) = timecode on the glove, sourced from the wireless jam sync from Hand Engine. This is a string of numbers representing hours mins seconds and frames, however, the colon is missing compared to Timecode (master) (1 column)

          4. Timer (device) = a millisecond counter of the device from startup should just increment in 8/9ms intervals (1 column)

          5. Timecode (master) = Hand Engine timecode (1 column)

          6. Solved joint angles (labeled) (60 columns)

      2. Metadata File .json
        1. File Naming Convention: <Performer number><Hand Side>_<Performer Name>Meta.json
        2. A metadata file is created for each hand (Left and Right) of the performer
        3. The metadata file contains the following information:
          {
            "source"
            "hash"
            "takeName"
            "startDate"
            "startTime"
            "performerName"
            "performerNumber"
            "profileName"
            "calibrationId": 0,
            "side": "Left",
            "gloveType": "RevE",
            "gloveSN": "00001369",
            "gloveFW": "01.01.01",
            "dongleFW": "01.01.02",
            "fileName": "P1L_Adam.csv",
            "outputType": "express",
            "masterTimeCodeSource": "System",
            "fpsRate": 60
          }

      13. Playing back raw recordings

      After recording hand data you are also able to play back the animation in Hand Engine.

      1. Click on the glove connection dropdown and select Add file source.



        If you already had a calibration loaded, the hand animation will begin playing as soon as the CSV file is loaded. Otherwise, continue to the following step.
      2. Find the location where the recorded files are saved and select the raw sensor data CSV file matching the hand side you are working with.
      3. Go to Advanced Calibration and navigate to the Hand Training tab, then click on Import Calibration. 
      4. Find the location where the recorded files are saved and open the Cal_XXX.json file matching the hand side you are working with.

      5. Once the calibration has been loaded you will need to click on Train. The hand animation will begin straight away, the animation will be playing in a loop. 

      WARNING

      Hand Engine is unable to import calibration files containing only an Express Calibration. Poses will need to be captured in order to be able to click on Train. 

       



      14. Remapping into a custom character

      Remapping is the function of assigning the bones of the hand asset within Hand Engine to drive the correct bones on the asset you are streaming hand data on to in another software. 

      This task will need to be completed prior to Step 14: Streaming hands to other software

       

      Setting Up Bind Pose for remapping FBX

      In order to remap the Hand Engine output to your custom character, you must first create an FBX that has just your target character in it and that character is in a full T-Pose that is saved at the first frame in the FBX.

      The character must be in a full T-pose, with each hand in an L-Pose or Paddle-Pose. The fingers of the character should align with a major axis of the scene/FBX.

      Please note this FBX is used only for extracting bone hierarchy and rotation order data from your character. Once the remapping process is complete, you will be able to stream data from Hand Engine to any FBX/scene containing that character.

      The key features for each hand are:

      1. The palm is facing down and is parallel to the ground plane

      2. The fingers are pointing straight out and are parallel both to each other and to the ground plane, and are aligned with a major axis of the scene/FBX e.g. the X-axis in the screenshots below.

      3. If setting the T-Pose in an L-Pose, the thumb is pointing straight forward and is both perpendicular to the fingers and parallel to the ground plane, and is aligned with one of the other major axes of the scene/FBX, e.g., the Z-axis in the screenshots below. The thumbnail should be facing towards the body, i.e. the opposite direction to which the fingers are pointing.

      Example of correct bind L-Pose

       

      Creating a remap profile in Hand Engine

      1. Open the remapping tool by going to Windows -> Remapping
      2. Select the character FBX you have set up as per above instructions:
      3. Once the file has been selected, please select the root of the left hand from the left drop-down menu.
      4. Hand Engine will detect the remainder of the bones on the hand hierarchy and list them below. 


        This is how the Hand Engine asset is mapped. Please use this as a guide for assigning the bones of your asset to the bones of the hand in Hand Engine:



      5. The right-hand column contains the hand bones of the asset within Hand Engine. Please select the associated bone to match your character's hand bones from this list. 
      6. Once this has been completed, you can adjust the save name of this mapping at the top right-hand side:
      7. To save this remapping, click on submit at the bottom of this window:


      If you need to make any adjustments to this remapping, you can make the changes and click update at the bottom. It will overwrite the previous saved remap:

       

      Assigning remap profile to a hand

      1. Please connect gloves and perform a calibration
      2. Click on Remapping Target in the hand setup box, and select your recently added remap. Make sure to add it for each hand.
      3. Once this has been done for each hand, the hands are now ready to stream outside of Hand Engine to your software of choice. Please see Streaming hands to other software for further instructions.


      15. Streaming hands to other software

      Please ensure you have completed Remapping to a character prior to streaming. 

      Please follow the steps in the following knowledge base guides to install the correct live link plugin for the software you are running:

      Maya: https://get.stretchsense.com/knowledge/streaming-from-hand-engine-to-maya-doc-5021

      MotionBuilder: https://get.stretchsense.com/knowledge/streaming-from-hand-engine-to-motionbuilder-doc-5014

      Unreal Engine: https://get.stretchsense.com/knowledge/streaming-from-hand-engine-to-ue4-unreal-doc-5012

      Unity: https://get.stretchsense.com/knowledge/streaming-from-hand-engine-to-unity-doc-5011

      Character & Remapping Setup:

      Please make sure you have created an FBX in the correct bind pose at frame one and used this to remap the bones of the hands as per Section 13: Remapping to a character.

      Turn on streaming in Hand Engine

      1. Make sure you have assigned a remap profile to each hand as per Section 13: Remapping to a character.
      2. In the focus performer view, check the TCP port is correct (Default 9000 forhand #1 and increments from there)
      3. Press the button to the right of the TCP port to turn the TCP stream on
      4. When the TCP stream is active, the switch will say "On"

      Setting up Livelink in your software of choice

      Please follow these sets of instructions for Maya, MotionBuilder, Unreal Engine and Unity to connect the Livelink in each software suite:

      Maya: https://get.stretchsense.com/knowledge/streaming-from-hand-engine-to-maya-doc-5021

      MotionBuilder: https://get.stretchsense.com/knowledge/streaming-from-hand-engine-to-motionbuilder-doc-5014

      Unreal Engine: https://get.stretchsense.com/knowledge/streaming-from-hand-engine-to-ue4-unreal-doc-5012

      Unity: https://get.stretchsense.com/knowledge/streaming-from-hand-engine-to-unity-doc-5011

        Xsens MVN Streaming:

        1. Please set up and turn streaming on in Hand Engine 2.0 as per the above instructions
        2. Please follow steps 4 onwards of this guide to complete: 

        Streaming Hand Engine -> MVN -> Unreal Engine

        Please note that the default network streamer port in MVN (9763) can conflict with the default port Hand Engine listens on for a remote recording trigger from MVN. 


        For the above set-up, please create two network streamers in MVN:

        1. Port 9764: Select Position+Orientation(Quaternion), Character Meta Data, Scaling Data, Send Finger Tracking Data and Send Object Tracking Data. 

        2. Port 9763: Select only "Network Sync XML" for the Datagram selection. This will transmit a recording trigger to Hand Engine

        When connecting the MVN Live Link in Unreal Engine, please set the port to 9764 (as per the above setting) 

         

        1. Unreal Engine Stream:
        2. Hand Engine Trigger:

         


        1. Batch Processing data recorded to SD Card

        1. Obtain the takes recorded to your SD card and copy them to your PC
          1. For Mocap Supersplay: Remove the SD card and use a card reader to copy files
          2. For Mocap Fidelity:
            1. Turn the glove off
            2. Hold the power button on the glove until the LED light turns a solid green
            3. Plug in a micro USB cable between glove and PC
            4. The SD card storage will show up as a drive on your PC.
        2. Drag and drop, or copy and paste all take folders from the SD card on top of the take folders that have been saved to your local PC.
          This will merge the locally recorded and SD card data into the same take folders. 

          once you have merged each take folder, they will appear to contain the following (Note, this example only has data from a single left-hand glove):


          Please make sure the take folder contains the files as shown above. It is important that you have the raw data files (CSV), Calibration file (CAL_*Performer*.json) and meta data files.
        3. In Hand Engine 2.0, open the batch processing tool in the Windows menu:
        4. Select the input folder. This will be the folder that contains all the takes you wish to process as shown above.
        5. Select the output folder, this will be where the Batch Processing tool will save all your take FBX files once complete.
        6. Click on Export.
        7. The tool will initialize and start processing all the takes in the input folder you selected:
        8. Once complete, your output folder will have populated with a folder for each take processed, and within that folder will contain up to 3 files:
          1. FullBody FBX
          2. Left hand FBX (Hand only)
          3. Right hand FBX (Hand only)