Unreal Engine (UE4 & UE5) - Streaming from Hand Engine 2

This document will take you through the steps of connecting to Unreal Engine with a Hand Engine data stream.

Requirements

• StretchSense Hand Engine version 2.2.2-SDK or earlier.

• Unreal Engine 4.24 to 4.27: https://www.unrealengine.com/en-US/

• Unreal Engine 5.0, 5.1 and 5.2 are also supported. 
  
  • Note below steps may differ slightly for UE5.0 and UE5.1

• Unreal Hand Engine Plugin: Download the most up-to-date Plugins for Unreal Engine from the Downloads Section of your StretchSense Account at http://www.stretchsense.com/my-account  

• Operating System: Windows 10

Setting up your Unreal Project

1. Download and Install the StretchSense Hand Engine and Unreal
   NOTE: for this example, we are using version UE4.24.3

2. Launch Hand Engine and calibrate the gloves.

3. Once the gloves are setup,  enter your TCP Port number and set the switch to On. Check that the Streaming Status indicator in the hand status bar pane has turned green

4. Next, we need to add the plugin to the Unreal Engine. To do this, first, navigate to the Plugins folder you have downloaded from your http://www.stretchsense.com/my-account   (see image below), and find the plugin directory corresponding to the version of UE you are running.

    

5. Open the folder corresponding to your version of Unreal, then copy the folder labeled MocapProLiveLink to the directory:

    <Unreal Installation Directory>\UE_4.2X\Engine\Plugins\Animation

   

6. Next, we will create a new UE4 project. In New Project Categories select Games > Blank > Default Settings. then click Create Project.
   
   

   

    

   

    

7. Once the project is loaded we need to enable the LiveLink plugins in your new project. From the Edit menu, select Plugins.

   

    

8. In the Plugins menu search “Live Link” and enable Live Link and Mocap Pro LiveLink, then restart your project when prompted.

   

    

9. Next, open the LiveLink Menu from Window > Live Link

   

    

10. From the Source dropdown in the Live Link menu select Mocap Pro Glove, then set the IP Address and TCP Port settings as specified in Hand Engine

    NOTE: The default value for Address is 127.0.0.1 (= localhost), i.e., you are running Hand Engine on the same PC that you are running Unreal. If you have bound Hand Engine to a different IP address (e.g. if you are running Hand Engine and Unreal on different computers on the same LAN), use this IP address instead for Host.

    NOTE: This step must be repeated if you have toggled Streaming to Off and then back to On in Hand Engine.

    

     

11. The Source Status will appear as Receiving and the Subject Name will show a hand (left or right) along with the Actor Name from Hand Engine.

    NOTE: If you are streaming pre-recorded data, you will need to make sure to press play in Hand Engine for the subject name and role to show up in Unreal.

    

     

12. From the Content Browser select Import and load the Hands asset. The Hands asset can be found in the Downloads Section of your StretchSense Account at http://www.stretchsense.com/my-account. The download is titled StretchSense FBX Files.
    
    Note: If you wish to stream to a metahuman or other character, please refer to the video at end of this article. If streaming onto any asset other than the StretchSense hands, a remapping function will need to be performed in Hand Engine.

    

13. In the Content Browser, right-click the newly imported Skeletal Mesh and select Create > Anim Blueprint.

    

     

14. Open the Anim Blueprint that was created, right-click on the empty space and select Live Link Pose.

    

     

    Connect a LiveLink Pose to the Output Pose. Select the source hand from the Live Link Subject Name dropdown, then compile the object. 

    

     

15. Drag the Anim Blueprint into the scene and start the scene in Play mode, the hand in the scene will now be driven from the Hand Engine data stream.

     

    

     

NOTE: If you would like the hand to update in Editor mode rather than Play Mode. Click on the Details tab of the Anim BluePrint you have dragged into the Level and under the Skeletal Mesh section make sure the tickbox for Update Animation in Editor is set to true (see image below).

Video Tutorial: Streaming to a MetaHuman Character in UE (2 min)

If you are streaming a custom rig, for example, a MetaHuman, you can use Hand Engine's remapping tool. The MetaHuman FBX mesh used for remapping in Hand Engine is available in the Downloads Section of your StretchSense Account at My Account - StretchSense under Software Downloads.

Perform a remapping function in Hand Engine following this tutorial (Link).

Once you have completed remapping, please turn the remap switch on.

This will ensure that Hand Engine is driving the correct bones on your character in Unreal Engine.

Disclaimer, Intellectual Property, and Licensing

Revision Summary