Earlier this year I was approached by s1t2 to work as Motion Capture Technician on a project made in collaboration with Spinifex and the Australian Defense Force.
S1T2's optitrack system had seen little acton in the years following its use for their project that debuted at Adobe Max in 2016.  When I defered my studies to take join the team as a junior 3d artist on beyond the stars I had recently completed my "Botley c-45" short film in which I had used an Opti-track system and was excited to make contiue applying what I had learned. However the time was not right as without a practical application, the purchase of the software silences was unfeasible. 

In March of 2020 I recieved a phone call from head of realtime Jack to inform me that he was finally granting my wish. 

s1t2 had been commissioned by Spinifex to create a VR training simulation for use by the Australian Defense Force to teach instructors how  to better spot infringements during live fire exercises.
In order to capture this we would be given access to four Australian Defence Force Commandos and their firearms for a day. 

And the shoot would be happening in 9 days time.

27th may. SUITS ARRIVE.

it's alive!

With such a tight turn around local suppliers of motion capture gear were caught off guard by a studio requiring 4 suits with with 37 markers for each suit as well as additional ones to track their firearms. 

We ended up with barely enough markers to get us through, but were unable to source cross-shaped mounts for them to get us over the line.

 Additionally, we had no diea what the firearms were meant to look like and had difficult gettign photos before the day.

Thankfully  I owned a 3D printer.

my first prototype marker

my 3d printed marker collection

I modeled up some solutions in Fusion 360 and began printing. However with the timeline so tight we were also fighting against my printers own print times. Despite being really happy with my designs for the barrel clip and clamp, I was only able to produce a single set as we needed to focus on the X-markers for the suits. 

Limited marker available wasn't the only problem that required a bespoke solution.  The drills were all focused on the correct way to enter/breach a room, with variant techniques depending on if there is or isn't a door, and if a grenade is to be thrown.

 So we needed a door, but one that would not obscure any of the markers in the scene, and we needed a grenade.
Stu's real fake door and our temporary rifle.
my bright idea for a temporary grenade.
Despite getting a chuckle my Kirby grenade was impractical and was replaced by a very impressive prop also made by Stu which had a pin which our performers could interact with. during the drills.

The next few days were dense with constant re-calbiration and resettign of the space as we attempted to maximise the useable space using only 6 optitrack track cameras (2 less than what is the recommended minimum) and understand the pipeline for streaming direction into Unreal Engine and recording with Sequencer.

While stressful, getting your bosses to dress up like power rangers and dance around took the edge off.

JK, Nat, Chris and cisco get down.

streaming to a paragon character in ue4

With as much set up, calibration and testing under our belt as possible. We spent the night getting in the suits and running mock exercises to ensure a smooth shoot day and attempt to predict any pitfalls or hurdles. 

We would have been certainly court marshaled for some of the catastrophic infringements that evening.

Chris, Myself, stu, nat and jack. seal 1 team 2.

And then it was shoot day.  Of which we weren't allowed to photograph because it would have been 'espionage'.

Whatever that means.

we recorded and categorized ~170 performance takes in a single 8 hour shoot. Successfully streaming the performance in motive for the performers as well as the Unreal Engine "Danger Room" on a different screen which allowed us to see the rigged characters interacting with their Unreal Environment. 

It was amazing seeing trained military specialists do their things and reduce it down to base motion data. We've all seen plenty of military dudes in videogames but seeing how their spatial awareness and how tightly their formations were executed made it feel closer to a dance than a training drill. 

4th june. shoot day.

After shoot day I was on clean up for a week exclusively in Motive. As we were recording performances directly into sequencer Technical Artist was hired to worked with me on IK solutions, to avoid a conventional clean up workflow with Motion Builder.

And that was a wrap for me!

S1T2 and Spinifex worked together on the final build of the project, if any further information of images get the green light to be shared I'll post them here.

I've had to privelage to work on some crazy projects but the experience of being trusted with such expensive equipment and to work with Australian Defense Force was definitely a career highlight. 

The muffin in the above photo was blueberry by the way. 

Thanks for your time!
Back to Top