Nvidia brings AI perception to ROS developers

Nvidia has unveiled initiatives to deliver a suite of perception technologies to the Robot Operating System (ROS) developer community.

The chipmaker says these initiatives will “reduce development time and improve performance for developers seeking to incorporate cutting-edge computer vision and AI/ML functionality into their ROS-based robotics applications”.

Nvidia and Open Robotics, the organisation which maintains ROS, have entered into an agreement to accelerate ROS 2 performance on Nvidia’s Jetson edge AI platform and GPU-based systems and to enable seamless simulation interoperability between Open Robotics’s Ignition Gazebo and Nvidia Isaac Sim on Omniverse.

The Jetson platform is widely adopted by roboticists across a spectrum of applications. It is designed to enable high-performance, low latency processing for robots to be responsive, safe, and collaborative.

Open Robotics will enhance ROS 2 to enable efficient management of data flow and shared memory across GPU and other processors present on Jetson. This will significantly improve the performance of applications that have to process high bandwidth data from sensors such as cameras and lidars in real-time.

In addition to the enhancements for deployment of robot applications on Jetson, Open Robotics and Nvidia are working on plans to integrate Ignition Gazebo and Nvidia Isaac Sim.

Nvidia Isaac Sim already supports ROS 1 & 2 out of the box and features an all-important ecosystem of 3D content with its connection to popular applications, for example Blender and Unreal Engine 4.

Ignition Gazebo brings a decades-long track record of widespread use throughout the robotics community, including in high-profile competition events such as the ongoing Darpa Subterranean Challenge.

With the two simulators connected, ROS developers can easily move their robots and environments between Ignition and Isaac Sim to run large scale simulations and take advantage of each simulator’s advanced features, such as high-fidelity dynamics, accurate sensor models, and photorealistic rendering to generate synthetic data for training and testing of AI models.

Brian Gerkey, CEO of Open Robotics, says: “As more ROS developers leverage hardware platforms that contain additional compute capabilities designed to offload the host CPU, ROS is evolving to make it easier to efficiently take advantage of these advanced hardware resources.

“Working with an accelerated computing leader like Nvidia and its vast experience in AI and robotics innovation will bring significant benefits to the entire ROS community.”

Software resulting from this collaboration is expected to be released in the spring of 2022.