NVIDIA opens Cosmos 3 checkpoints for interactive simulation across autonomous driving, robotics and controllable video. The November 5 release gives developers four open-weight, 16-billion-parameter variants of the company’s Cosmos 3 Nano world model.

The models cover multi-camera road scenes, synchronized LiDAR generation and two-arm manipulation. They are available under NVIDIA’s OpenMDW1.1 license, which permits commercial and noncommercial use subject to its terms, according to the official release on Hugging Face.

The four checkpoints target distinct simulation jobs:

• Transfer-Auto generates offline driving data across multiple sensors.
• Sim-Auto streams camera and LiDAR observations for closed-loop driving.
• Sim-Transfer converts simulator controls into photorealistic RGB video.
• Sim-Bimanual predicts robot-camera observations from actions.

Related Research

• Agility Robotics Starts Nasdaq Trading With $465 Million in Gross Capital
• OpenAI Physical Engineering Push Adds Mazda Research and 0.875 BenchCAD Score

NVIDIA Opens Cosmos 3 Checkpoints Across Four Workflows

Cosmos 3 is a generative world-model family designed to produce future visual observations from an initial scene and control inputs. The new post-trained checkpoints narrow that general capability into workflows that developers can evaluate without training a model from the base checkpoint.

The release is more specific than a conventional text-to-video update. Each model accepts control signals suited to a physical system, such as vehicle trajectories, simulator buffers or robot joint commands, and produces visual observations that can be consumed by data pipelines or interactive simulations.

An accompanying Cosmos 3 research paper describes a shared framework spanning camera navigation, robot manipulation, controllable video transfer and multi-view autonomous-vehicle simulation. NVIDIA has made code and checkpoints available through GitHub and Hugging Face, turning the paper’s system into testable artifacts.

All four releases list 16 billion trainable parameters. NVIDIA’s technical documentation reports generation memory figures of 58 GiB in BF16, 44 GiB in FP8 and 31 GiB with FP8 plus model offloading, though actual deployment requirements depend on the checkpoint, resolution and surrounding simulator.

Transfer-Auto and Sim-Auto Generate Camera and LiDAR Streams

Cosmos3-Nano-Transfer-Auto targets offline synthetic driving data. It can generate synchronized video for as many as 11 camera views, including fisheye cameras, while optionally producing aligned RGB and LiDAR streams. Controls can come from a World Scenario Map and a text prompt describing appearance.

The NVIDIA technical guide says Transfer-Auto supports 30-frame-per-second output for clips of up to 400 frames at 480p or 720p. Longer sequences can be produced by generating successive chunks rather than one uninterrupted sample.

Cosmos3-Nano-Sim-Auto is the causal, distilled variant intended for streaming. Instead of generating an entire sequence offline, it produces successive camera and LiDAR chunks in a small number of denoising steps. The design is meant to support closed-loop testing, where an agent’s action changes the next observation.

The checkpoint is not a complete driving simulator. NVIDIA’s documentation says users still need an external simulation runtime and a renderer for the World Scenario Map. Physics, traffic state and agent logic remain outside the generative model, which supplies the sensor-facing visual layer.

Sim-Transfer and Bimanual Split Physics From Visual Prediction

Cosmos3-Nano-Sim-Transfer keeps the same division of labor for general simulation. A conventional simulator maintains state and physics, then passes control buffers such as depth, segmentation, edges or blurred imagery to Cosmos. The model combines those inputs with text instructions to generate RGB observations.

That approach lets teams preserve deterministic control over geometry while using a generative model for appearance. It could help vary lighting, materials and scene texture without asking the model to invent the underlying physics, although developers must still measure whether generated frames preserve task-critical details.

Cosmos3-Nano-Sim-Bimanual applies the framework to two-arm robots. It predicts the next video observation from an initial image plus robot and camera actions, and NVIDIA lists support for YAM and AgiBot embodiments. The reference setup was tested on one GB200 system, which is a reported configuration rather than a universal minimum.

For robotics teams, the practical attraction is action-conditioned video that can extend scarce demonstration data or provide an observation model inside a simulator. The checkpoint does not replace a control policy: it predicts what the camera should see after an action, while a separate policy chooses the action.

Open Weights Leave Safety and Policy Performance Unanswered

The open-weight release improves inspectability and gives researchers reproducible files to test. It does not establish that synthetic sensor streams improve downstream driving or manipulation policies, and NVIDIA has not published independent evidence that these checkpoints are reliable enough for safety-critical decisions.

Useful evaluation will need to examine temporal consistency, cross-camera alignment, LiDAR geometry and whether rare hazards survive generation. For bimanual tasks, researchers will also need to test contact-rich actions, occlusions and error accumulation over long rollouts rather than judging short clips on visual quality alone.

Cosmos 3 therefore arrives as infrastructure for controlled experiments, not proof of autonomous-system readiness. The four specialized checkpoints make the framework easier to benchmark across driving and robotics, while leaving deployment teams responsible for simulator validation, policy testing and real-world safety cases.

Feature image: NVIDIA headquarters in Santa Clara, photographed by Coolcaesar and edited by Jacek Halicki, via Wikimedia Commons under CC BY-SA 4.0. Cropped to 16:9; the photograph shows the company’s headquarters, not the Cosmos models.