Digital Twins

A digital twin does not fit inside a headset

A complete industrial model runs into millions of polygons and expects a serious machine underneath it. A standalone headset, meanwhile, is a phone strapped to your face. Between the two you usually pick one of two sacrifices: strip the model down until it fits — losing on the way the very detail that justified building a twin — or stream the headset's own view from the workstation, and discover that the slightest network delay is paid for in discomfort.

That second point is worth dwelling on, because it is not a question of raw power. When the video stream is the headset's view, every turn of the head has to travel to the server and back before the image follows. The world lags behind the gaze, and the inner ear does not forgive that.

The L3DS approach: stream a screen, not a viewpoint

L3DS does not broadcast what the user sees. It broadcasts a large stereoscopic screen placed in space, which turns to stay in front of them. The user is not looking through the stream: they are looking at an object, inside a scene their headset keeps drawing on its own.

That single shift changes everything. Head rotation stays local, handled by the headset's compositor at its own rate — the stream no longer carries responsibility for it. A network delay then shows up in the content of the screen, where the eye tolerates it, rather than in the stability of the world, where it does not. Latency that would make a direct view unusable becomes perfectly acceptable.

What sits underneath

• A binary protocol, compact and versioned, described in a single reference file from which both sides are generated — C# for Unity, Python for Blender. A change to the format cannot leave the two implementations out of step.
• WebRTC for transport. Camera pose goes out at roughly 60 Hz over an unreliable channel: a dropped position beats a late one.
• One image for both eyes, left eye in the left half, split again at display time by the shader — one stream rather than two to keep in sync.
• Quality that follows motion. The client measures its own linear and angular speed and attaches them to every pose it sends; the server decides on its own to push a high-quality frame once the gaze settles. Bits are spent where they can be seen.
• Two modes. VR for full immersion, and Portal in passthrough — the screen becomes a window opened into the real room, with a transparent background and an adjustable height above the floor.

Between which tools

The server — the side holding the heavy scene and rendering it — is either a Unity application or a Blender add-on. The client is a Unity application on a standalone headset, or a WebXR-capable browser. In other words: the scene stays where it was authored, in the tool it lives in, and it is the view that travels.

A protocol is only worth something if it is open

We want to release L3DS as open source. This is not a communications gesture: it is the only outcome consistent with what this piece of software is. An exchange protocol that stays closed is not a protocol, it is a private format — it is worth exactly as much as the number of programs that speak it, and nobody writes a connector against a specification they cannot read.

The work has been heading that way from the start. The binary format fits in one readable reference file, from which the Unity and Blender implementations are generated: that is precisely the shape you give a specification meant to be picked up by other people, not internal code. A third engine, a fourth authoring tool, plug in by writing their own reading of that file.

The same conviction drives our work on OpenUSD: open formats are what keeps a 3D model from becoming a prisoner of the tool that produced it.