HelixWorld 1.0 is a real-time interactive audio-visual world model from Noiz AI, with researchers from HKUST, Tsinghua, CMU, and Google DeepMind.
Give it an image and a prompt. Walk forward or turn around — picture and sound update together from the same Transformer. The spatial field (azimuth, distance, reverb) turns with the camera. Audio is not a soundtrack laid on afterwards.
Weights, inference code, and the technical report ship in the coming weeks. Open a feature request if you want something on the roadmap.
| HelixWorld 1.0 | |
|---|---|
| Roaming / camera navigation | Yes |
| Real-time | 24 FPS |
| Continuous generation | > 3 minutes |
| Joint audio-video | 48 kHz stereo, same Transformer |
| Spatial sound field | Follows viewpoint |
| Weights and code | Coming soon |
Genie 3, WorldPlay, LingBot-World and others roam visually. HelixWorld's difference is native audio-video, plus a spatial field that turns with the camera.
Training code may follow inference. Dataset is not planned for 1.0. Subscribe to releases.
Four stages. Details, figures, and ablations ship with the report.
spatial AV data → joint Transformer + action
→ causal rollout (KV cache, self-generated history)
→ few-step distillation (trajectory + DMD) for realtime
- Spatial AV data. First-person real-world video (on-location sound) plus game-engine captures with known geometry and listener pose. Fake stereo, post-hoc scores, and narration are dropped.
- Joint generation + action. Audio and video keep their own latents, enter one Transformer, and predict the next picture and sound from the current state and the user's action.
- Causal rollout. Bidirectional video models can look ahead; interaction cannot. Converted to a causal generator with KV cache, then trained on its own generated history.
- Realtime. Trajectory distillation + DMD cut denoising to a handful of steps. Action, decode, and AV output run as a pipeline.
1.0 is real-time audio-video interaction. Next:
| Tracker | |
|---|---|
| Multi-agent | #5 |
| Multi-person presence | #6 |
| Long-horizon consistency | #7 |
| Autonomous evolution | #8 |
| Games / interactive film / education | #10 |
@misc{helixworld2026,
title = {HelixWorld: A Real-Time Interactive Audio-Visual World Model},
author = {{Noiz AI}},
year = {2026},
howpublished = {\url{https://github.com/NoizAI/HelixWorld}},
note = {Weights and code forthcoming}
}Code is Apache 2.0. The weight license will be published with the checkpoint.
Related work from the team: AudioX, AudioX-Turbo, YuE.
