1. StretchSense Knowledge Base
  2. Streaming to Other Software

Streaming from Hand Engine to UE4 (Unreal) (DOC-5012)

Requirements

Setting up your Unreal Project

  1. Download and Install the StretchSense Hand Engine and Unreal
    NOTE: for this example, we are using version UE4.24.3

  2. Launch Hand Engine and setup hand
    See Hand Engine User Guide (DOC-5046) for glove calibration instructions

  3. Once the hand is trained,  in the Streaming box on the left of the screen, enter your TCP Port number and set the switch to On. Check that the Streaming Status light in the Performer pane title bar (the right traffic light indicator) has turned green.

    NOTE: If you have already created a LiveLink connection to a hand model in UE (as described in the following steps), and you toggle Streaming to Off, you will need to create a new instance of LiveLink in UE after you toggle Streaming to On again.

  4. Next, we need to add the plugin to the Unreal Engine. To do this, first, navigate to the Plugins folder you have downloaded from your http://www.stretchsense.com/my-account   (see image below), and find the plugin directory corresponding to the version of UE4 you are running.

     

  5. Open the folder corresponding to your version of Unreal, then copy the folder labeled MocapProLiveLink to the directory:

     <Unreal Installation Directory>\UE_4.2X\Engine\Plugins\Animation

  6. Next, we will create a new UE4 project. In New Project Categories select Games > Blank > Default Settings. then click Create Project.

     

     

  7. Once the project is loaded we need to enable the LiveLink plugins in your new project. From the Edit menu, select Plugins.

     

  8. In the Plugins menu search “Live Link” and enable Live Link and Mocap Pro LiveLink, then restart your project when prompted.

     

  9. Next, open the LiveLink Menu from Window > Live Link

     

  10. From the Source dropdown in the Live Link menu select Mocap Pro Glove, then set the IP Address and TCP Port settings as specified in Hand Engine

    NOTE: The default value for Address is 127.0.0.1 (= localhost), i.e., you are running Hand Engine on the same PC that you are running Unreal. If you have bound Hand Engine to a different IP address (e.g. if you are running Hand Engine and Unreal on different computers on the same LAN), use this IP address instead for Host.

    NOTE: This step must be repeated if you have toggled Streaming to Off and then back to On in Hand Engine.

     

  11. The Source Status will appear as Receiving and the Subject Name will show a hand (left or right) along with the Actor Name from Hand Engine.

    NOTE: If you are streaming pre-recorded data, you will need to make sure to press play in Hand Engine for the subject name and role to show up in Unreal.

     

  12. From the Content Browser select Import and load the Hands asset. The Hands asset can be found in the Downloads Section of your StretchSense Account at http://www.stretchsense.com/my-account. The download is titled StretchSense FBX Files.

    Note: If you wish to stream to a metahuman or other character, please refer to the video at end of this article. If streaming onto any asset other than the StretchSense hands, a remapping function will need to be performed in Hand Engine.

  13. In the Content Browser, right-click the newly imported Skeletal Mesh and select Create > Anim Blueprint.

     

  14. Open the Anim Blueprint that was created, right-click on the empty space and select Live Link Pose.

     

    Connect a LiveLink Pose to the Output Pose. Select the source hand from the Live Link Subject Name dropdown, then compile the object. 

     

  15. Drag the Anim Blueprint into the scene and start the scene in Play mode, the hand in the scene will now be driven from the Hand Engine data stream.

     

     

NOTE: If you would like the hand to update in Editor mode rather than Play Mode. Click on the Details tab of the Anim BluePrint you have dragged into the Level and under the Skeletal Mesh section make sure the tickbox for Update Animation in Editor is set to true (see image below).

 

 

Video Tutorial: Streaming to a MetaHuman Character in UE (2 min)

If you are streaming a custom rig, for example a MetaHuman, you can use Hand Engine's remapping tool. The MetaHuman FBX mesh used for remapping in Hand Engine is available on the Downloads Section of your StretchSense Account at My Account - StretchSense under Software Downloads.

Perform a remapping function in Hand Engine following this tutorial (Link).

Once you have completed remapping, please turn the remap switch on.

This will ensure that Hand Engine is driving the correct bones on your character in Unreal Engine.

 

Disclaimer, Intellectual Property, and Licensing

Revision Summary

Revision

Date

Notes

12

2022.03.01

Embedded metahuman video. Added step 4 (Remapping)

11

2021.11.01

Updated links for HS

10

2021.10.04

Updates images for HE 1.4

9

2021.09.03

Updated for UE 4.27. Plugin 1.0.5 and higher is compatible with UE 4.27

8

2021.07.27

Updated images for HE 1.2.1

7

2021.03.24

Updated for Hand Engine 1.1.0

6

2021.03.08

Updated for Hand Engine 1.0.1

5

2020.10.02

Updated for Beta4

4

2020.09.21

Updated for Beta3.5

3

2020.08.21

Updated for Beta3

2

2020.07.14

Updated related docs, fixed typos, added disclaimer text

1

2020.07.09

Initial version