NVIDIA Advances Physical AI With Accelerated Robotics Simulation on AWS

Field AI is building robot brains that enable robots to autonomously manage a wide range of industrial processes. Vention creates pretrained skills to ease development of robotic tasks. And Cobot offers Proxie, an AI-powered cobot designed to handle material movement and adapt to dynamic environments, working seamlessly alongside humans.

These leading robotics startups are all making advances using NVIDIA Isaac Sim on Amazon Web Services. Isaac Sim is a reference application built on NVIDIA Omniverse for developers to simulate and test AI-driven robots in physically based virtual environments.

NVIDIA announced at AWS re:Invent today that Isaac Sim now runs on Amazon Elastic Cloud Computing (EC2) G6e instances accelerated by NVIDIA L40S GPUs. And with NVIDIA OSMO, a cloud-native orchestration platform, developers can easily manage their complex robotics workflows across their AWS computing infrastructure.

This combination of NVIDIA-accelerated hardware and software — available on the cloud — allows teams of any size to scale their physical AI workflows.

Physical AI describes AI models that can understand and interact with the physical world. It embodies the next wave of autonomous machines and robots, such as self-driving cars, industrial manipulators, mobile robots, humanoids and even robot-run infrastructure like factories and warehouses.

With physical AI, developers are embracing a three computer solution for training, simulation and inference to make breakthroughs.

Yet physical AI for robotics systems requires robust training datasets to achieve precision inference in deployment. Developing such datasets, however, and testing them in real situations can be impractical and costly.

Simulation offers an answer, as it can significantly accelerate the training, testing and deployment of AI-driven robots.

Harnessing L40S GPUs in the Cloud to Scale Robotics Simulation and Training

Simulation is used to verify, validate and optimize robot designs as well as the systems and their algorithms before deployment. Simulation can also optimize facility and system designs before construction or remodeling starts for maximum efficiencies, reducing costly manufacturing change orders.

Amazon EC2 G6e instances accelerated by NVIDIA L40S GPUs provide a 2x performance gain over the prior architecture, while allowing the flexibility to scale as scene and simulation complexity grows. The instances are used to train many computer vision models that power AI-driven robots. This means the same instances can be extended for various tasks, from data generation to simulation to model training.

Using NVIDIA OSMO in the cloud allows teams to orchestrate and scale complex ‌robotics development workflows across distributed computing resources, whether on premises or in the AWS cloud.

Isaac Sim provides access to the latest robotics simulation capabilities and the cloud, fostering collaboration. One of the critical workflows is generating synthetic data for perception model training.

Using a reference workflow that combines NVIDIA Omniverse Replicator, a framework for building custom synthetic data generation (SDG) pipelines and a core extension of Isaac Sim, with NVIDIA NIM microservices, developers can build generative AI-enabled SDG pipelines.

These include the USD Code NIM microservice for generating Python USD code and answering OpenUSD queries, and the USD Search NIM microservice for exploring OpenUSD assets using natural language or image inputs. The Edify 360 HDRi NIM microservice generates 360-degree environment maps, while the Edify 3D NIM microservice creates ready-to-edit 3D assets from text or image prompts. This eases the synthetic data generation process by reducing many tedious and manual steps, from asset creation to image augmentation, using the power of generative AI.

Rendered.ai’s synthetic data engineering platform integrated with Omniverse Replicator enables companies to generate synthetic data for computer vision models used in industries from security and intelligence to manufacturing and agriculture.

SoftServe, an IT consulting and digital services provider, uses Isaac Sim to generate synthetic data and validate robots used in vertical farming with Pfeifer & Langen, a leading European food producer.

Tata Consultancy Services is building custom synthetic data generation pipelines to power its Mobility AI suite to address automotive and autonomous use cases by simulating real-world scenarios. Its applications include defect detection, end-of-line quality inspection and hazard avoidance.

Learning to Be Robots in Simulation

While Isaac Sim enables developers to test and validate robots in physically accurate simulation, Isaac Lab, an open-source robot learning framework built on Isaac Sim, provides a virtual playground for building robot policies that can run on AWS Batch.

Because these simulations are repeatable, developers can easily troubleshoot and reduce the number of cycles required for validation and testing.

Several robotics developers are embracing NVIDIA Isaac on AWS to develop physical AI, such as:

Aescape’s robots are able to provide precision-tailored massages by accurately modeling and tuning onboard sensors in Isaac Sim.
Cobot has used Isaac Sim with its AI-powered cobot, Proxie, to optimize logistics in warehouses, hospitals, manufacturing sites, and more.
Cohesive Robotics has integrated Isaac Sim into its software framework called Argus OS for developing and deploying robotic workcells used in high-mix manufacturing environments.
Field AI, a builder of robot foundation models, uses Isaac Sim and Isaac Lab to evaluate the performance of its models in complex, unstructured environments across industries such as construction, manufacturing, oil and gas, mining and more.
Standard Bots is simulating and validating the performance of its R01 robot used in manufacturing and machining setup.
Swiss Mile is using Isaac Sim and Isaac Lab for robot learning so that wheeled quadruped robots can perform tasks autonomously with new levels of efficiency in factories and warehouses.
Vention, which offers a full-stack cloud-based automation platform, is harnessing Isaac Sim for developing and testing new capabilities for robot cells used by small to medium-size manufacturers.