Unreal Engine & Vive Trackers - Comprehensive Guide for Set Up with Hand Engine

This document should take you through the steps of how to integrate MoCap Pro gloves with Vive trackers in Unreal Engine.

Requirements

Software

StretchSense Hand Engine 1.1.0 or higher: Login to your account page at http://www.stretchsense.com/my-account or email support@stretchsense.com for details
Unreal Engine 4.25.4: https://www.unrealengine.com/en-US/
Unreal Hand Engine Plugin: Download the most up-to-date Plugins for Unreal Engine from the Downloads Section of your StretchSense Account at http://www.stretchsense.com/my-account
Steam Desktop Application with SteamVR Installed (https://store.steampowered.com/app/250820/SteamVR/)
Operating System: Windows 10

Hardware

VR Capable PC or Laptop (please see your headset's recommended specifications)
SteamVR Compatible Headset (HTC Vive, Valve Index, Varjo)
Vive Trackers 1.0 or 2.0 (You can use 1 or 2 trackers depending on how many gloves you are using)

Please note: To attach the Vive Trackers to the Glove via the Velcro Strap you must have adhesive velcro tape on the bottom of the Vive Tracker)
Base Stations (we recommend at least 2x Base Station 2.0)

Video Tutorial (≈ 6 min)

Setting up SteamVR and Vive Trackers

Download and Install the Steam Desktop client and SteamVR. Launch SteamVR and let it run the first time setup.

NOTE: For this example, we are using SteamVR Version 1.16.10 with an HTC Vive Pro, 3x Base Station 2.0s, and 2x Vive Tracker 2.0s
Connect your Headset and Base Stations, conduct the Room Setup and Pair your Vive trackers as per the device's instructions. Once the Vive Trackers are paired right-click on one of the trackers and click Manage Vive Trackers (see image below).

In the Steam VR settings select Manage Vive Trackers (see below)

For the Vive Trackers, you have Connected leave the Tracker Role set to Held in Hand. Change the Hand to the Glove you will be physically attaching the Tracker to e.g. Left Hand for a Left Glove. Click Close and also close Steam VR Settings.
Leave Steam VR running in the background and begin to set up your Unreal Project.

Setting up your Unreal Project with Hand Engine

Download and Install the StretchSense Hand Engine and Unreal Engine

NOTE: for this example, we are using version UE 4.25.4
Launch Hand Engine and setup hand
Once the hand is trained check the TCP port is correct (Default 9000 for hand #1 and increments from there). Press the button to the right of the TCP port to turn the TCP stream on.

NOTE: If you have already created a LiveLink connection to a hand model in UE (as described in the following steps), and you toggle Streaming to Off, you will need to create a new instance of LiveLink in UE after you toggle Streaming to On again.
Next, we need to add the plugin to the Unreal Engine. To do this, first, navigate to the Plugins folder you have downloaded from your http://www.stretchsense.com/my-account (see image below), and find the plugin directory corresponding to the version of UE4 you are running.
Open the folder corresponding to your version of Unreal, then copy the folder labeled MocapProLiveLink to the directory:

<Unreal Installation Directory>\UE_4.2X\Engine\Plugins\Animation
Next, we will create a new UE4 project. In New Project Categories select Games > Virtual Reality > Default Project Settings. then click Create Project.
Once the project is loaded create a new level and select default (see image below).
Click on the small triangle next to the Play button and then from the menu that is displayed, select the VR Preview option.
Then we need to enable the LiveLink plugins in your new project. From the Edit menu, select Plugins. In the Plugins menu search “Live Link” and enable Live Link and Mocap Pro LiveLink, then restart your project when prompted.

NOTE: Make sure the SteamVR plugin is also enabled. It should be by default for the VR starter project.
Next, open the LiveLink Menu from Window > Live Link
From the Source dropdown in the Live Link menu select Mocap Pro Glove, then set the IP Address and TCP Port settings as specified in Hand Engine

NOTE: The default value for Address is 127.0.0.1 (= localhost), i.e., you are running Hand Engine on the same PC that you are running Unreal. If you have bound Hand Engine to a different IP address (e.g. if you are running Hand Engine and Unreal on different computers on the same LAN), use this IP address instead for Host.

NOTE: This step must be repeated if you have toggled Streaming to Off and then back to On in Hand Engine.
The Source Status will appear as Receiving and the Subject Name will show a hand (left or right) along with the Actor Name from Hand Engine.

NOTE: If you are streaming pre-recorded data, you will need to make sure to press play in Hand Engine for the subject name and role to show up in Unreal.
From the Content Browser select Import and load the Hands.fbx asset. The Hands.fbx asset can be found in the Assets folder in the installation directory of Hand Engine.
In the Content Browser, right-click the newly imported Skeletal Mesh and select Create > Anim Blueprint.
Open the Anim Blueprint that was created, right-click on the empty space and select Live Link Pose.

Connect a LiveLink Pose to the Output Pose. Select the source hand from the Live Link Subject Name dropdown, then compile the object.
If you would like to set up a second glove and Hand follow steps 9 to 14 for the second hand noting to change the port number to match the second glove.

For example, the default port number for Glove 2 in Hand Engine is 9001. Therefore when selecting the second MoCap Pro Glove in the Live Link Window change the port number to 9001.

unreal-plugin-port-9001

Setting Up Unreal Engine with SteamVR and Steam Trackers

Create a new Pawn Blueprint Class (see images below) and give it a name e.g. VRPawn. See here for instructions:
Open up the new Pawn Blueprint. Select the VRPawn(self) that you created in the Components tab. Next, in the Details panel under the Pawn setting, set the Auto Possess Player from Disabled to Player 0.
Go to the Component section of the Viewport tab. From there add a Scene component and rename to VRCameraRoot and secondly, and Camera component as a child of VRCameraRoot (see below).
Next, while in the Event Graph tab drag off the Event Begin Play node to show the Executable Actions list. In the list search for the Set Tracking Origin node and click on it to add it to the Event Graph. The Set Tracking Origin node has two options, Floor Level and Eye Level. For a standing experience, you will need to leave the Origin of the Set Tracking Origin node to the default of Floor Level.
Go back to the Component section of the Viewport tab. From there add a MotionController component as a child of DefaultSceneRoot (see below). It is good practice to rename the MotionController to reflect what hand it will be tracking.

NOTE: In the Details panel under the Motion Controller setting, the Motion Source to Left (see image below)

NOTE: If you want to track a Vive Tracker to track a Right Hand Glove then the Motion Source should be set to Right.
Go back to the Component section of the Viewport tab. From there add a SkeletalMesh component as a child of MotionController (see below).

In the Details panel under the Animation setting, set the Anim Class to the AnimBlueprint you created (see image below)

In the Details panel under the Mesh setting, set the Skeletal Mesh to the Hands_L_Hand skeletal mesh (see image below).
Compile and Save the Pawn Blueprint. Navigate back to the Level you created.

NOTE: If you are also setting up a second glove and Vive Tracker then follow steps 5 to 7 to set up another Motion Controller component. The final VR pawn structure will look as follows and then Compile and Save:
Drag the Pawn Blueprint from the Content Browser to a level, making sure that it is placed at 0,0,0 in the level.
Play the Level and there you go You have finger tracking and the hand position is tracked by the Vive Tracker!