Vicon Shogun Post 1.8 - Merging StretchSense Data

Vicon Shogun Post 1.8 includes a custom script that can easily merge hand recordings performed in Hand Engine with body MoCap captured in Shogun Live.

Pre-requisites:

• Hand Engine 2.X or later
• Vicon Shogun Post 1.8 or later
• Latest StretchSense Script from Vicon
  • https://www.vicon.com/software/models-and-scripts/stretchsense-scripts/
• Latest VSS files from Vicon
  • https://www.vicon.com/software/models-and-scripts/vss/

Motion Capture Setup and Data Management:

• Set the Performer name in the Hand Engine exactly the same as the performer name in Shogun Live. This will make sure that the automated merging of hand and body data will be successful. 
• Ensure that you have set up timecode sync between Shogun Live and Hand Engine, or that the two software are operating on the same PC with the same clock:
  • https://get.stretchsense.com/knowledge/hand-engine-user-guide-2.0#Timecode
• Before recording, ensure that the Master Timecode in Hand Engine is live. 
  
  Example: Shogun Live operating on same PC as Hand Engine:
  

• Ensure that you have set up remote triggering from Shogun Live to Hand Engine. This will ensure that recordings are started and stopped at the same time:
  
  • https://get.stretchsense.com/knowledge/hand-engine-user-guide-2.0#RemoteTrigger
    
    Setting in Shogun Live
    
    Setting in Hand Engine - Settings menu
    

• Set the Hand Engine recording folder to the same as Shogun Live. This will ensure that all data recorded will be in the same place
• Ensure the frame rate is the same for the Hand data from Hand Engine and the body MoCap from Shogun Live. If you are not sure, you can check this by importing the recording FBX from Hand Engine into Shogun Post and checking the frame rate at the bottom right-hand side. 
• Make sure the time code for the Hand Recordings closely matches the body MoCap recordings to ensure alignment of the hand and body data. 

Setting the Take Folder:

For ease of data ingest, it is suggested to set the same capture path for both Hand Engine and Shogun Live. 

If you set the capture path in both Shogun and Hand Engine to:
G:\MOCAP\StretchSense\ViconCapture
The following are produced in that folder:
From Shogun: An MCP file named MyTake_001.mcp (Example)
From Hand Engine: A take folder named after the date, within that, subfolders named MyTake_001 etc. (Example), and within these take folders, the FBX files required for the data merge. 

NOTE: Hand Engine will create a take folder of the current date. You will need to copy the take folders from within this and move them into the root folder of the Vicon Takes. See the example below for clarification.

Example:
The root of the capture folder:

Within Hand Engine Take folder:

Shogun Post Script - Automatic Merge

Provided all the above prerequisites have been met, and the data is organized as per the above instruction, Follow these steps to merge the data.

1. Open Shogun Post 1.8 and load the MCP file of the take containing the solved actor Vicon body skeleton(s)
   
   
2. In the script tab, run the following:
   ImportStretchSenseFBXFiles.hsl;
   
   
3. All hand data FBX files with the correct naming scheme will be imported and merged to the respective performers' skeleton in the scene. 

Shogun Post Script - Manual Merge

If the data from Hand Engine has a different performer name or does not meet the above naming schema, you can still manually merge the left and right-hand data onto the performers' skeleton in Shogun Post.

1. Open Shogun Post 1.8 and load the MCP file of the take containing the solved actor Vicon body skeleton(s)
2. Set the current subject to the performer you wish to merge that hand data onto.
3. In the script tab, run the following:
   ImportStretchSenseFBXFiles.hsl;
4. A file browser window will open, please select the Left and Right hand .FBX files you wish to merge. 
5. Shogun Post will then merge the Hand data onto the solved actor vicon body skeleton. 
   
   

Troubleshooting

Hands are merging at odd angles (Pre-rotation set incorrectly)

If the hands, once merged, are offset in rotation from the wrist, you may need to edit the pre-rotation offsets in the importStretchSenseFBXFile.hsl script.  

1. Assess the pre-rotation by loading the Vicon .MCP and merging the hand data as per above instructions
2. Select the wrist of each hand and check the pre-rotation in the attributes  window. 
   
3. Adjust the pre-rotations in Shogun Post until the hands are aligned correctly with the wrist . Take note of these values (X, Y, Z)
   
   
   
4. Open importStretchSenseFBXFile.hsl  using a text editor such as Notepad++
5. Insert these changes into this part of the script. Note the first set of rotations is for the Left hand, and the second (After the "else") is for the right hand.
   The order is as follows:
   
   setProperty"Pre_Rotation" X Y Z -onMod $SS_HandName;
   
   Note in the below script section, the numbers have been updated to match the above images. 
   
   

   // Remove translation offset and set pre rotation to  

   // orient StretchSense hand relative to Vicon hand  

   select $SS_HandName;  

   selectProperty Translation;  

   cutKeys -all;  

   setProperty "Translation.X" 0.000000 -d  -onMod $SS_HandName;  

   if( $side == "L" )  

   {  

   setProperty "Pre_Rotation" 180.000000 0.000000 90.00 -onMod $SS_HandName;  

   }  

   else  

   {  

   setProperty "Pre_Rotation" 0.000000 0.000000 -90.0 -onMod $SS_HandName;  

6. Save the script  (You may be asked to save as administrator)
7. Open a new scene, and import the Vicon MCP.
8. Run the ImportStretchSenseFBXFiles; script 
9. Select the hand data files
10. Check if the hands are aligned as expected with the wrist.

Timecode misalignment between Hand and Body MCP files:

If there is a mis-match in the timecode between Hand Engine and Shogun Live at the time of recording, you are still able to perform the merge. Please follow these steps to remedy:

1. In Shogun Post, load the Left and Right-hand FBX files (Separately) with the following import settings:
   
2. Import FBX as Skeleton
   
3. Select the clip
   
4. In the attributes section, select the advanced attributes
   
   
   
5. Set SMPTE_Offset to 0:
   
6. Select the SolvingBone branch:
   
7. Export FBX with these settings:
   
8. Start a new scene in Shogun Post, and import the vicon body MCP, perform the same actions above to set SMPTE_Offset to 0 before running the ImportStretchSenseFBXFiles.hsl; script.
9. After merging, the clip timecode can be restored in the advanced attributes.

The frame rate between Hand and Body data does not match

If the frame rate at the time of recording was not set the same in Shogun Live and Hand Engine, your body and hand recordings may have mis-matching frame rates. To remedy this, please follow the below instructions.

1. In Shogun Post, Load your Vicon body recording MCP and note the frame rate shown at the bottom right hand side
   
2. In Shogun Post, start a new scene and load the Left and Right-hand FBX files (Separately) with the following import settings:
   
3. In the Script Editor, for this example, run the following script. Note: Adjust the resample encoding and frame rate to suit your body MCP
   
   selectByType SolvingBone;
   resample "NTSC" 119.88; 
   
   
   
   
   
4. Export FBX with these settings:
   
5. You should now be ready to run the merge script as per the above instructions.