Motion capture into AR

Overview

Working in the Centre for Creative and Immersive XR, I built this project in two halves. The first played pre-recorded motion capture onto a character viewed through Apple Vision Pro. The second took a live performer's movement from an optical mocap stage and drove a character that a co-located viewer could see in real time through Meta Quest 3.

It sits at the intersection of the things I work with most — mocap stages, virtual production, and headset interaction design — and it draws directly on a background most XR designers don't have: I both perform and direct on the volume, so I shape the capture as a piece of staging, not just data.

What I did

• Directed and performed the captured motion myself — blocking the action on the volume and shooting it as a scene.
• Operated capture on a Vicon Shogun optical system and cleaned the resulting data.
• Set up real-time previsualisation during capture using Autodesk MotionBuilder and Unreal Engine 5.
• Delivered the pre-recorded performance to Apple Vision Pro, and streamed the live skeleton into Meta Quest 3 so a co-located viewer sees the performance mapped into their own space.
• Tuned the live loop for low latency and legibility, the two things that make or break real-time mocap-to-AR.

Performance as design

Before games I trained and worked as an actor, and I bring that to the stage: knowing how to block a movement, hold an eyeline, and time a beat changes the quality of a capture before any cleanup happens. It's the same instinct a designer uses to stage a scene and read how a space performs for the person inside it — which is why I see motion capture and game design as the same craft viewed from two sides of the camera.

Selected views

// Swap in cleaner stills or a longer clip whenever you like — files live in assets/web/.