The challenge in accelerating robotics AI isn’t always the artificial intelligence itself, but the painstaking work of preparing virtual environments for training. NVIDIA is now applying its NemoClaw agents to automate this process, inspecting, labeling, and configuring 3D scenes for use in platforms like NVIDIA Isaac Sim and NVIDIA Isaac Lab.
This workflow uses NVIDIA Omniverse Libraries to convert scenes created in Blender into a simulation-ready OpenUSD format, addressing a critical bottleneck for robotics developers. “The extra work required to get a 3D scene into that state is laborious, time consuming, and frequently out of scope,” explain Max Bickley and Ashley Goldstein, outlining a system designed to deliver a.
NemoClaw Agents Address Simulation Prep Bottlenecks
NemoClaw agents are now capable of autonomously identifying and rectifying validation issues within 3D scenes intended for robotics simulation, streamlining a process previously dominated by manual inspection. This automated correction extends beyond simple file conversion; the agents address specific deficiencies, as one engineer reported, “I fixed validation issues automatically.” The ability to pinpoint and resolve these issues represents a shift from merely preparing a file to ensuring a scene adheres to a defined.
This approach addresses a critical bottleneck in robotics AI development, where the time spent preparing virtual environments often exceeds the time spent training the AI itself. The system’s validation process is not a pass/fail assessment, but a detailed check against specific requirements, including correct collision meshes, meaningful materials for simulation, properly placed sensors, and clean USD exports. The orchestration agent, powered by either Codex or Claude, coordinates specialized NemoClaw subagents, routing tasks and determining when the preparation is complete.
This is achieved through the SimReady Blender addon, which validates scenes against target profiles, flagging any failures for human review before proceeding. NVIDIA’s investment in this area extends to hardware; the DGX Spark system is positioned as ideal for local prototyping and development of these NemoClaw subagents. Beyond the technical capabilities, this agentic engineering approach aims to redefine the workflow for robotics simulation.
The goal is to move beyond manual, repetitive tasks and towards a system where agents can autonomously prepare and validate digital twins for physical AI systems. This is about integrating reasoning, tool execution, and validation into a repeatable process. The use of semantic labels, for example, adds meaning to the scene, while ovphysx and ovrtx libraries contribute to physical realism and visual testability, respectively. The prepared scenes are designed to be directly trainable within NVIDIA Isaac Sim or Isaac Lab, completing the cycle from virtual world creation to AI model training.
Agentic Workflow: From Blender to SimReady OpenUSD
The coordinated operation of specialized agents is now automating the preparation of 3D scenes for robotics simulation, moving beyond simple inspection to actively authoring simulation-relevant data. This workflow, detailed by NVIDIA researchers, centers on converting scenes originating in Blender into a simulation-ready format utilizing OpenUSD, a foundational element for physically accurate virtual environments. The system doesn’t merely identify missing data; it actively creates collision shapes and physics properties, rendering preflight views to assess the scene’s readiness before validation.
A key component of this process is the integration of NVIDIA Omniverse Libraries, providing the tools necessary for these “NemoClaw” subagents to modify and validate scenes. OpenUSD operations establish the scene’s structure, while libraries like ovphysx handle physics properties and ovrtx generates visual preflight views. This modular approach allows for focused, specialized tasks, coordinated by a general-purpose agent like Codex or Claude, which manages the overall objective of creating a simulation-ready world.
The workflow explicitly defines a progression from input, a Blender scene, to output, a USD-based environment, with a clear destination of either NVIDIA Isaac Sim or NVIDIA Isaac Lab, the company says. The process addresses a specific set of requirements, moving beyond a vague request to “make this scene better” to a defined objective with measurable outcomes. For instance, making an object “grabbable” necessitates coordinated updates to its semantic label, rigid-body configuration, and collision geometry, a task handled by the interconnected subagents.
This level of detail extends to validation, where the SimReady validation agent utilizes established profiles to assess whether the scene meets the necessary standards for accurate simulation. If validation fails, the resulting report is automatically converted into a task list for corrective agents, streamlining the debugging process. NVIDIA’s investment in this area extends to standardization; SimReady Foundation defines standards and validation profiles for simulation-ready USD content.
This standardization is critical for ensuring consistency and reliability across different simulations, allowing robots to train in environments that accurately reflect real-world physics and interactions. Beyond the software components, NVIDIA’s broader strategy includes hardware acceleration. The company’s CUDA Quantum platform and DGX Quantum systems provide the infrastructure for quantum computing services, while partnerships with companies like Quantinuum, QuEra Computing, and IonQ demonstrate a commitment to exploring the intersection of quantum computing and artificial intelligence, according to NVIDIA.
Recent collaborations, such as the link between Quandela quantum processors and AI via NVQLink, and the use of quantum-enhanced AI on NVIDIA GPUs to steer molecules, highlight the company’s expanding portfolio in advanced computing technologies. A 2026-07-28 report detailed a quantum calibration model functioning across six qubit modalities, further demonstrating this commitment.
The ultimate goal, according to the researchers, is to create a seamless pipeline from 3D scene creation to robot training, eliminating the bottlenecks that currently hinder the development of physical AI systems. “The point of using NVIDIA Omniverse Libraries in an agent workflow is to integrate the tools agents need to build SimReady worlds,” they state, emphasizing the importance of providing agents with the necessary tools to act effectively. This approach promises to accelerate the development of more robust and adaptable robots, capable of learning and operating in complex, real-world environments.
Codex & Astra Orchestrate Multi-Agent Scene Inspection
Codex and Claude, general-purpose agents, can now identify the specific steps required to prepare a 3D scene created in Blender for use in robotics simulations, a capability previously demanding significant manual effort. This automated recognition is a foundational element, but the system extends beyond simple identification to actively modify scenes, authoring collision shapes and physics properties essential for realistic simulations.
The workflow detailed uses NVIDIA’s NemoClaw architecture to deploy specialized subagents, each focused on a discrete task within the broader scene preparation process, utilizing open-source agent harnesses like Hermes, OpenClaw, or LangChain. These subagents, configured with NVIDIA Nemotron models for vision, reasoning, and tool use, operate under the coordination of either Codex or Claude, which translates developer objectives into actionable tasks and reviews the results.
In a configuration employing Astra and Hermes, Codex directs Astra to decompose the overarching goal into dependencies, then deploys Hermes subagents through NemoClaw to execute those tasks, the firm reports. This division of labor allows for a granular approach to scene preparation, addressing specific requirements such as identifying missing simulation metadata and rendering visual preflight views. The system’s architecture moves beyond simply automating tedious tasks, establishing a framework for agentic engineering where software tools are directly manipulated by AI to achieve a defined outcome, NVIDIA reports.
The process begins with a clearly defined objective, input Blender scene, desired output as a USD-based simulation-ready world, destination within either Isaac Sim or Isaac Lab, and validation criteria encompassing visual preflight and SimReady validation, providing the orchestration agent with sufficient structure to manage the workflow. This structured approach contrasts with earlier methods, where a broad request to “make this scene better” lacked the precision needed for automated execution.
The initial phase involves a Hermes inspection subagent inventorying the scene, detailing collections, hierarchy, materials, cameras, lights, and identifying potential robot targets, obstacles, and other relevant elements, then structuring that information for use by subsequent agents. Codex then coordinates NemoClaw to activate the appropriate subagents, in this instance utilizing Blender MCP to analyze the “Junk Shop” scene and assess its components. A Blender add-on bridges scene data to Omniverse Libraries via gRPC, enabling rendering and physics capabilities to directly update the Blender viewport.
Semantic labeling is then performed, assigning task-relevant classes to scene elements, with the agent inferring labels from object names, hierarchy, shape, and context, while also flagging uncertainties for human review; the system recently identified nine labels requiring review. This process utilizes ovrtx, an Omniverse Library tool, to inspect the scene and apply the semantic segmentation necessary for robotics simulation readiness.
The system’s reliance on USD as a “contract” ensures data consistency and interoperability, with semantic labels adding meaning, sensors defining perception, and ovphysx establishing physical properties. SimReady validation then confirms the scene meets the necessary criteria for training within Isaac Sim or Isaac Lab. NVIDIA, founded in 1993 and headquartered in Santa Clara, has established itself as a key player in this emerging field, with ten patent families and four publications in the last twelve months demonstrating ongoing investment in agentic workflows.
Omniverse Libraries Enable Automated Physics & Rendering
NVIDIA Omniverse Libraries are now integral to a workflow automating the assignment of physics properties to 3D scenes, a process previously demanding significant manual effort. Specialized subagents, orchestrated by the Hermes agent harness and deployed through NVIDIA’s NemoClaw, use these libraries to inspect scenes and configure elements for realistic simulation, addressing a critical bottleneck in robotics development. The system moves beyond simply creating visually appealing environments to ensuring those environments behave predictably under simulated physical forces.
The coordinated operation of these subagents relies on OpenUSD as a foundational element, establishing a shared scene structure that facilitates data exchange and consistency. Once a scene is authored in USD, agents can inspect individual components, add relevant metadata, and validate requirements before passing the world forward to training environments like NVIDIA Isaac Sim or NVIDIA Isaac Lab. This approach avoids the pitfalls of fragile, one-off exports that often require extensive debugging.
Astra, a component of the system, connects these requirements across subagents, determining which checks are necessary before the workflow progresses. SimReady validation then evaluates the resulting assets against a defined simulation profile, ensuring they meet the necessary criteria for training. “The USD authoring agent uses Omniverse Libraries to preserve hierarchy, transforms, materials, labels, physics metadata, and sensor definitions,” according to the documentation detailing the workflow.
Codex, or Claude, serves as the orchestration agent, defining the main objective, input, desired output, destination, and validation criteria, and coordinating the NemoClaw subagents. In a demonstration using a scene, NemoClaw inventoried objects, materials, and scene structure. This level of automation addresses a challenge frequently encountered by robotics simulation engineers: the laborious and time-consuming task of preparing 3D scenes for training.
NVIDIA’s investment in agentic systems extends to tools like CUDA-Q, accelerated algorithm libraries for quantum-classical computing launched in March 2023, and CUDAqx, launched in October 2024. These platforms, alongside partnerships with Quantinuum, QuEra Computing, and others, demonstrate a broader commitment to accelerating computational workflows. Recent reports indicate a quantum calibration model working across six qubit modalities, and a link between quantum processors and AI via NVQLink, established with Quandela in September 2026. This standardization is crucial for ensuring reproducibility and scalability in robotic training.
Blender MCP Provides Agent Access to Scene Inventory
NVIDIA’s NemoClaw agents now access scene data directly through Blender’s Model Context Protocol, or MCP, a controlled tool interface that allows agents to inspect objects, collections, transforms, materials, cameras, lights, and scene metadata without relying on screenshots or manual exports. This inventory is structured and shared as a common context for subsequent subagents, enabling coordinated action and reducing redundant analysis. The Hermes inspection subagent returns its findings to Codex, which then orchestrates the necessary tasks, using NVIDIA Omniverse Libraries as the toolkit for modification, rendering, validation, and preparation of the virtual world.
According to a demonstration of the system, nine labels required review, highlighting the ongoing need for human oversight even with advanced automation. This agent’s work isn’t simply about naming objects; it’s about providing the simulation engine with the information it needs to understand how those objects should behave.
The goal, as articulated by the developers, is a scene that meets the simulation contract, a set of defined requirements for a functional virtual environment, rather than merely a file export. NemoClaw then orchestrates a subagent to utilize the SimReady Blender addon, validating the scene against target SimReady profiles. Any failures in validation are flagged for human review before proceeding, ensuring a level of quality control. The system reports its ability to autonomously resolve many common issues.
See today’s quantum computing news on Quantum Zeitgeist for the latest breakthroughs in qubits, hardware, algorithms, and industry deals.
