Workflow · Unreal Engine and TouchDesigner

Move your hands.
The rig follows.

A webcam tracks your hands, TouchDesigner turns the movement into DMX values, and the lights of a virtual stage in Unreal Engine respond live.

Watch on YouTubeSee the signal chain

Rehearse the light show before a single fixture is hung.

The signal chain

Four links, all running live on one machine.

  1. 01 · Input

    Webcam

    An ordinary camera pointed at you. No gloves, no sensors.

  2. 02 · Tracking

    Hand tracking

    TouchDesigner follows the position of your hands in the image.

  3. 03 · Protocol

    DMX values

    The movement becomes the same channel data a lighting desk would send.

  4. 04 · Output

    Unreal Engine rig

    The moving heads of the virtual stage read those channels and follow.

One stage, driven live

The stage in the Unreal Engine editor, with truss, LED screens and fixtures

Built in Unreal Engine

Truss, three LED screens and rows of moving heads, laid out like the real rig.

White beams crossing above the LED screens

Crossed beams

Both hands apart, beams open.

Warm orange beams fanning out from the stage

Warm fan

Colour and angle change on the fly.

Green beams and gobo spots on the stage floor

Green wash

The same rig, a different gesture.

The full breakdown
goes on the channel.

The node setup in TouchDesigner and the DMX patch in Unreal Engine, step by step.

Open the channel