Hand Engine 2.0 or Later - User Guide

 1. Setting Up Your MoCap Pro Glove Hardware

In this section you will find:

• USB Bluetooth Dongle Connection
• Wired USB Connection
• Pairing a glove to USB Bluetooth Dongle

USB Bluetooth Dongle

Each MoCap Pro glove uses one USB Bluetooth Dongle. Out of the box, these are already paired to each glove. 

1. Please connect the USB Bluetooth Dongle to any USB Type-A port on your PC (Please see Tips below for more information)
2. The USB Bluetooth Dongle will have a solid green LED showing. This means that it is waiting for a connection to its paired MoCap Pro Glove.
3. Power on the MoCap Pro Glove by pressing the power button on top of the glove once.
4. A slow flashing blue LED on the glove means that it is not currently connected to a USB Bluetooth Dongle (One second on, one second off)
5. The glove will automatically connect to its paired dongle. Once connected, the glove LED will change to a flashing blue LED (One quick flash every second) and the USB Bluetooth Dongle will have a solid blue LED.
6. Windows will detect the USB Bluetooth Dongle and automatically apply a COM port number to that dongle.  (You may check these in the Windows Device Manager)
7. You are now ready to create and stage a performer and connect to Hand Engine.

Wired USB Connection

You may connect your MoCap pro gloves to your PC using a Micro USB cable in a wired configuration. This may be helpful when doing desktop capture sessions and a wireless connection is not required.

1. Please connect your glove to your PC using a Micro USB cable. If you are using a Fidelity glove, skip to step 3.
2. On the side of the SuperSplay glove electronics module, please change the switch from C (Charge) to D (Data). 
3. Power on the glove by pressing the power button once
4. One powered on, please press the power button on the glove twice (quickly)
5. The LED on the glove will change from blue to a quick flashing green color (One flash every second)
6. You are now ready to create and stage a performer and connect to Hand Engine.
   
   

Pairing USB Bluetooth Dongle

Out of the box, your glove should already be paired to its associated USB Bluetooth Dongle.

If your glove is no longer paired to the USB Bluetooth Dongle, or you have received a replacement dongle, please follow these instructions to re-pair.

1. Please turn all gloves off in the vicinity.
2. Make sure the USB Bluetooth Dongle is connected and is showing a solid green LED.
3. Press and hold the button on the top for approximately 10 seconds while the dongle is plugged in.
4. The LED on the USB Bluetooth Dongle will turn solid red.
5. Power on the glove and place it within 10cm of the USB Bluetooth Dongle.
6. The dongle LED will change to solid green, and then to solid blue once the glove has connected.
   
   

---

 

2. License Key Activation

In this section you will find:

• Automatic Activation
• Manual Activation
• Floating License Activation

The first time Hand Engine is launched it will open on the activation screen requesting a license key. The key can be obtained from your account on StretchSense Downloads. 

If you do not already have an account you can sign up here  Create an Account - StretchSense.

Contact our Support team at support@stretchsense.com if you have any issues or if your license key is not in your account.

Automatic Activation:

Copy and paste your license key into the required box, then click on Activate.

If Hand Engine fails to activate, follow the information from the error pop-up. Contact our Support team at support@stretchsense.com if you are unable to activate your license.

Manual Activation:

Click on the manual activation option, paste your license key, and click on 'Save Activation Request'. 

Hand Engine will generate an .XML file, which will need to be sent to our Support team at support@stretchsense.com, then we will send you back an Activation Response .XML file.

Go back to Hand Engine and click on 'Set Activation Response' to load the file provided by our Support team, then click on Activate. 

Floating License Activation:

With the Floating License, you must first install the Floating Server on a computer on your network. Once it has been installed and activated, the server has a “pool” of “license leases” (i.e., the maximum number of instances of your app that can run at any given time Hand Engine can only run 1 instance at a time). Then you can install Hand Engine on as many computers as you want.

This also works over a VPN, so if you have other locations that require access to the Hand Engine license, you can point them towards the floating license server as long as they are on the same VPN network.

When you launch Hand Engine, the following happens:

1. Hand Engine tries to connect to the Floating Server installed on the network. If Hand Engine can successfully connect to the Floating Server, it requests a “license lease” from the Floating Server instance. One of two things will happen:
   
   a) If there's 1 “license lease” left in the “pool” then Hand Engine will get that “license lease”.
   
   b)If there are 0 “license leases” left in the “pool” then Hand Engine will not be able to start.
    
2. When you close Hand Engine, it releases that “license lease” back into the “pool” so that it can be used by other users in that network.
   
   

In Hand Engine, paste your license key into the appropriate box and click on 'Install floating license server here'

 A command prompt window will open and Windows will request for admin access twice during the installation.

Once the process is complete click on Activate to finish the activation process and open Hand Engine on the server PC.

When you close Hand Engine, the license lease will be released back to the server and available for another PC to open Hand Engine.

The floating license server is now installed on this PC.

Unstage a Performer

To unstage a performer, please de-select the check box next to that performer. You can select this box to bring the performer back to the stage.

 

---

4. Connecting Your Gloves to Hand Engine

in this section you will find the three methods for connecting your MoCap Pro gloves to Hand Engine:

• USB Bluetooth Dongle Connection
• Wired USB Connection
• WiFi Bridge App Connection

To connect to your gloves, please stage a performer, and next to the associated hand (Left or Right), click on the drop-down box to select the connection mode:

 

USB Bluetooth Dongle Connection

Please select the associated com port for your USB Bluetooth dongle and Hand Engine will connect the hand to that dongle. (Make sure your glove is turned on for data to stream)

Wired USB Connection

Please follow these instructions for connecting gloves in a wired configuration.

Serial Port Setup:

1. SuperSplay
2. Fidelity
   1. Open windows device manager, and select 

 

1. Please follow the below step depending on glove type:
   
   1. For SuperSplay: Please make sure the switch on your glove electronics module is set to "D" (Data), then power the glove on by momentarily pressing the power button. 
   2. For Fidelity: Power the glove on by momentarily pressing the power button. 

1. Plug the micro USB cable into your glove, and into your PC. 
2. Power the glove on, and then press the power button once momentarily. The LED light on the glove will change to a quick flashing green.
3. In the Hand Engine connection dropdown, select the correct USB connection to connect the glove:
   

WiFi Bridge App Connection

You can download the StretchSense Wifi Bridge application here:

https://play.google.com/store/apps/details?id=com.stretchsense.smartapp

Pair your gloves with the app and take note of the port numbers in use. The port numbers start from 9000 and increment for each additional glove connected. (Glove 1: Port 9000, Glove 2: Port 9001)

Please select Add network port and enter the port number the associated glove is set to on the WiFi bridge application. 

Once you have entered the correct port number, please click on Submit to connect.

---

 

5. Exporting and Importing Calibration

Hand Engine 2.0 has the ability to export and import calibrations for your performers. This can be very useful if moving between machines, or if you wish to keep a backup of your calibration.

Calibrations are saved as a .json file.

Exporting a Calibration:

1. Make sure a glove is connected and calibration has been completed (See Express Calibration  and Advanced Calibration)
2. Focus Performer and click on Advanced Calibration
3. Within the Hand Training tab, please click Export Calibration button 
   
4. Give your calibration file a name and save it somewhere convenient.

Importing a Calibration

1. Make sure a glove is connected and calibration has been completed (See Express Calibration  and Advanced Calibration)
2. Focus Performer and click on Advanced Calibration
3. Within the Hand Training tab, please click Import Calibration button 
   
4. Navigate to where your calibration files are saved and open the calibration file you require. 
5. Once the calibration has loaded all the poses, please select the poses you wish to use for calibration (Blend or Pose), or turn "Include Express Calibration" and click the Train button.
   
   

 

---

6. Express Calibration Capture

 

1. Make sure your glove is connected, the performer staged and data is coming through.
   

2. Click on the Calibrate Express button
   
3. Please perform hand motions as per the below video
   
   

4. Once you have completed the Express Calibration, you can start recording or streaming right away! or try the Advanced Calibration method.

---

7. Advanced Calibration - Pose Capture

---

8. Training the Hand

• Express
• Blend Pose Mode
• Key Pose Mode
• Tuned Express
• Express with Key Pose
• Blend Pose with Key Pose
• Tuned Express with Key Pose
   

Express Mode

If you have performed an express calibration. Hand Engine 2.0 will default the hand output to using the express mode.

If you have used an Advanced Calibration training method such as Blend, Pose, or Hybrid and wish to go back to the Express output mode:

Click on "Train" to set the output to Express Mode

Blend Pose Mode

Key Pose Mode

Hybrid Calibration Options

 Tuned Express

Tuned Express is the combination of Express calibration with some blend poses to improve the overall accuracy of the express calibration.

1. Perform an Express Calibration
2. Open the Advanced Calibration section
3. Capture poses you wish to use to improve upon the express calibration you have done

1. Select the Hand Training Tab
2. Make sure that "Include Express Calibration" is turned on
3. Select the poses you wish to use for the tuned express under the Blend Pose column
   
   
4. Click on Train and assess your calibration. 

Express with Key Pose

Express with Key Pose is the combination of express mode calibration and selecting key poses.

1. Open the advanced Calibration Section
2. Select the Hand Training tab
3. Ensure you have performed an Advanced Calibration and captured a set of poses.
4. Turn "Include Express Calibration" on
5. Deselect any poses that may be ticked under the Blend column.
6. Select the poses you wish to key under the pose column
   
    

    Important! - Selecting Key Poses for Hybrid Modes:
   

   Please select a minimum of two poses for this to operate correctly. The two poses need to be at opposite extremes of joint angle. You are able to set the key confidence of the opposing pose extremely low if you do not wish Hand Engine to key to that pose. 
   
   

   Example: I wish to key to the Fist pose, I will need to select the paddle pose and I will set the key confidence of Paddle to 5, Fist I will set key confidence to 15 as I wish to key into this pose.

     
   

7. Click on "Train" 
   
   
8. Test the sensitivity of the key poses, and adjust the Key Confidence higher or lower depending on how strongly you wish to lock into these poses. 
   
9. If you have changed any key poses or key confidence, please click the Train button again to update the calibration. 
   
10. All other hand motions besides the key poses will be based on the express calibration.

Blend Pose with Key Pose

Blend Pose with Key Pose is the combination of a blend base calibration and selecting key poses. 

---

12. Recorded File Information

Recordings done within Hand Engine will produce the following files within the recording location folder you have selected:

1. Calibration .json
   1. File Naming Convention: Cal_<Performer number><Hand Side>_<Performer Name>.json
   2. A calibration file is created for each hand (Left and Right) of the performer
2. Full Body FBX
   1. File Naming Convention: <Performer number>Fullbody_<Performer Name>.FBX.
   2. This .FBX contains the animation of both hands attached to a full skeleton.
3. Hands-Only FBX
   1. File Naming Convention: <Performer number><Hand Side>_<Performer Name>.FBX
   2. This .FBX contains the animation of one hand in a hand mesh.

2. Raw Data CSV
   1. File Naming Convention: <Performer number><Hand Side>_<Performer Name>.csv
   2. A raw data CSV is created for each hand (Left and Right) of the performer.
   3. File Contains:

      1. Raw capacitance data of sensors (16 columns)

      2. Raw IMU data (if applicable) (10 columns)

      3. Timecode (device) = timecode on the glove, sourced from the wireless jam sync from Hand Engine. This is a string of numbers representing hours mins seconds and frames, however, the colon is missing compared to Timecode (master) (1 column)

      4. Timer (device) = a millisecond counter of the device from startup should just increment in 8/9ms intervals (1 column)

      5. Timecode (master) = Hand Engine timecode (1 column)

      6. Solved joint angles (labeled) (60 columns)

3. Metadata File .json

1. 1. File Naming Convention: <Performer number><Hand Side>_<Performer Name>Meta.json
   2. A metadata file is created for each hand (Left and Right) of the performer
   3. The metadata file contains the following information:
      {
        "source"
        "hash"
        "takeName"
        "startDate"
        "startTime"
        "performerName"
        "performerNumber"
        "profileName"
        "calibrationId": 0,
        "side": "Left",
        "gloveType": "RevE",
        "gloveSN": "00001369",
        "gloveFW": "01.01.01",
        "dongleFW": "01.01.02",
        "fileName": "P1L_Adam.csv",
        "outputType": "express",
        "masterTimeCodeSource": "System",
        "fpsRate": 60
      }

Setting Up Bind Pose for remapping FBX

In order to remap the Hand Engine output to your custom character, you must first create an FBX that has just your target character in it and that character is in a full T-Pose that is saved at the first frame in the FBX.

The character must be in a full T-pose, with each hand in an L-Pose or Paddle-Pose. The fingers of the character should align with a major axis of the scene/FBX.

Please note this FBX is used only for extracting bone hierarchy and rotation order data from your character. Once the remapping process is complete, you will be able to stream data from Hand Engine to any FBX/scene containing that character.

The key features for each hand pose are:

1. The palm is facing down and is parallel to the ground plane

2. The fingers are pointing straight out and are parallel both to each other and to the ground plane, and are aligned with a major axis of the scene/FBX e.g. the X-axis in the screenshots below.

3. If setting the T-Pose in an L-Pose, the thumb is pointing straight forward and is both perpendicular to the fingers and parallel to the ground plane, and is aligned with one of the other major axes of the scene/FBX, e.g., the Z-axis in the screenshots below. The thumbnail should be facing toward the body, i.e. the opposite direction to which the fingers are pointing.

Example of correct bind L-Pose

 

Common Mistakes

Thumb is in a relaxed position. This will cause the thumb to be tucked in too close to the hand

Fingers are curled/in a relaxed position. This will cause the fingers to be over-rotated then creating a fist.

Fingers and thumb are correctly set, but the hand/arm is rotated relative to the major axis of the global scene. This will cause off-axis rotations of the fingers/thumb when curling them.

15. Streaming Hands to Other Software

---