• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to secondary sidebar
  • Skip to footer
  • Home
  • Subscribe
  • Your Membership
    • Edit Your Profile
  • Services
    • Advertising
    • Case studies
    • Design
    • Email marketing
    • Lead generation
    • Magazine
    • Press releases
    • Publishing
    • Sponsored posts
    • Webcasting
    • Webinars
    • White papers
    • Writing
  • Shop
    • My Account
    • Cart
  • About
    • Contact
    • Privacy
    • Terms of use
  • Events

Robotics & Automation News

Market trends and business perspectives

  • News
  • Features
  • Video
  • Webinars
  • White papers
  • Press releases
  • Featured companies
    • AMD Xilinx
    • BlueBotics
    • Elite Robot
    • RGo Robotics
    • SICK Sensor Intelligence
    • Vicor Power

Reinforcing the value of simulation: Teaching dexterity to a real robot hand

December 8, 2022 by Mark Allinson Leave a Comment

Nvidia researchers show how training in simulation enables the transfer of complex manipulation skills to a robot hand with project DeXtreme

The human hand is one of the most remarkable outcomes of millions of years of evolution. The ability to pick up all sorts of objects and use them as tools is a crucial differentiator allowing us to shape the world around us.

For robots to work in the everyday human world, the ability to deftly interact with our tools and the environment around them is critical. Without that capability, they will continue to be useful only in specialized domains such as factories or warehouses.

While it has been possible to teach robots with legs how to walk for some time, robots with hands have generally proven to be much trickier to control. A hand with fingers has more joints, and they must move in specific coordinated ways to accomplish a given task.

Traditional robotics control methods with precisely pre-programmed grasps and motions are incapable of the kind of generalized fine motor control skills that humans take for granted.

One approach to these problems has been the application of Deep Reinforcement Learning (RL) techniques that train a neural network to control the robot’s joints. With deep RL, a robot learns from trial and error and is rewarded for the successful completion of the assigned task.

Unfortunately, this technique can require millions or even billions of samples to learn from, making it almost impossible to apply directly to real robots.

Applying Simulation

Enter Nvidia’s Isaac robotics simulator, which enables robots to be trained inside a simulated universe that can run more than 10,000 times faster than the real world, and yet obeys the laws of physics.

Using Isaac Gym, an RL training robotics simulator, Nvidia researchers on the DeXtreme project taught this robot hand how to manipulate a cube to match a provided target position and orientation or pose. The neural network brain learned to do this entirely in simulation before being transplanted to control a robot in the real world.

Similar work has only been shown once before by researchers at OpenAI. Still, their work required a far more sophisticated and expensive robot hand, a cube tricked out with precise motion control sensors, and, last but not least, used a supercomputing cluster of hundreds of computers to train.

Democratizing Dexterity

The hardware used by the DeXtreme project was chosen to be as simple and inexpensive as possible to enable researchers worldwide to replicate our experiments. The robot itself is an Allegro Hand, which costs as little as 1/10th the cost of some alternatives, has four fingers instead of five, and has no moving wrist.

We can use off-the-shelf RGB cameras to track the cube with vision, which can be repositioned easily as needed without requiring special hardware, and the cube is 3D-printed with stickers affixed to each face.

DeXtreme is trained using Isaac Gym, which provides an end-to-end GPU-accelerated simulation environment for reinforcement learning. Nvidia PhysX simulates the world on the GPU, and results stay in GPU memory during the training of the deep learning control policy network.

As a result, training can happen on a single Omniverse OVX server. Training a good policy takes about 32 hours on this system, equivalent to 42 years of a single robot’s experience in the real world.

Not needing a separate CPU cluster for simulation means a 10-200x reduction in computing costs for training at current cloud rental rates. Because we can leverage Isaac Gym to train the model, training time and cost can be dramatically reduced.

Perception and Synthetic Data

In order for the robot to know the current position and orientation of the cube it’s holding, a perception system is needed. To keep costs low and leave open the potential for manipulation of other objects in the future, DeXtreme uses three off-the-shelf cameras and another neural network that can interpret the cube pose.

This network is trained using about 5 million frames of Synthetic Data generated using Omniverse Replicator and no real images whatsoever. The network learns how to perform the task under challenging circumstances in the real world.

To make the training more robust, we use a technique called domain randomization to change lighting and camera positions, plus data augmentation to apply random crops, rotation, and backgrounds.

The DeXtreme pose estimation system is very reliable and can perceive accurate poses even when the object in question is partly occluded from view, or the image has significant motion blur.

Real Robots are Still Challenging

One of the key reasons to use simulations is that training robots directly in the real world are riddled with various challenges. For example, robot hardware is prone to breaking after excessive usage, and experiments’ iteration cycles and turnaround time can be slow.

During our experiments, we often found ourselves repairing the hand after prolonged usage, for example, tightening the loose screws, replacing the ribbon cables, and resting the hand to cool down after running 10-15 trials.

Simulations allow us to sidestep many of these issues by training on a robot that doesn’t wear out and also provides the large diversity of data needed to learn challenging tasks. At the same time, because simulations can run much faster than in real time, the iteration cycle is massively improved.

When training in simulation, the most significant challenges are bridging the gaps between the simulations and the real world. To address this, DeXtreme uses domain randomization of the physics properties set in the simulator – changing object masses, friction levels, and other attributes at scale across over a hundred thousand simulated environments at once.

One interesting upshot of these randomizations is that we train the AI with all kinds of unusual combinations of scenarios, which translates to robustness when performing the task in the real world.

For instance, most of our experiments on the real robot took place with a slightly malfunctioning thumb due to a loose connection on the circuit board. Still, we were positively surprised that the policies transferred from simulation to the real world reliably regardless.

Sim-to-Real

Future breakthroughs in robotic manipulation will enable a new wave of robotics applications beyond traditional industrial uses. At the heart of the DeXtreme project is the message that simulation can be an incredibly effective tool for training complex robotic systems, even ones that need to handle environments with objects in continual contact with the robot.

We hope that by demonstrating this using relatively low-cost hardware, we can inspire others to use our simulation tools and build on this work.

For further details on the DeXtreme project, check out the paper and visit the project webpage.

For a further dive into simulators and how they can impact robotics projects, download the latest version of Omniverse Isaac Sim, read this blog that covers the topic, and learn about training your own reinforcement learning policies!

About the Authors

  • Gavriel State is a Senior Director for Simulation and AI at Nvidia, based in Toronto, where he leads efforts involving applications of AI technology to simulation systems and vice versa. Previously, Gavriel founded TransGaming Inc, and spent 15 years focused on games and rendering technologies.
  • Ankur Handa is a Research Scientist at Nvidia Seattle Robotics group led by Dieter Fox. Prior to that, he was a Research Scientist at OpenAI, and before that he was a Dyson Fellow at Imperial College London. He finished his PhD with Prof Andrew Davison at Imperial College London and did a two-year post-doc with Prof. Roberto Cipolla at the University of Cambridge. His papers have won Best Industry Paper Award at BMVC, in 2014 and have been the Best Manipulation Paper Award Finalist and Best Student Paper Award Finalist at ICRA 2019.
Print Friendly, PDF & Email

Share this:

  • Print
  • Facebook
  • LinkedIn
  • Reddit
  • Twitter
  • Tumblr
  • Pinterest
  • Skype
  • WhatsApp
  • Telegram
  • Pocket

You might also like…

Filed Under: Features, Science Tagged With: control, cube, data, dextreme, experiments, hand, hardware, isaac, learning, network, nvidia, project, real, researchers, robot, robotics, robots, simulation, simulations, task, time, train, training, work

Join the Robotics & Automation News community

Reader Interactions

You must log in to post a comment.

Primary Sidebar

Latest articles

  • Ad Blockers Could Be Helpful for Your Privacy
  • LG robots deliver Korean food from Atlanta restaurant
  • Recycleye raises additional $17 million in Series A funding
  • 5 Reasons Why You Should Invest in Tesla this 2023
  • Starship expands robotic delivery to more college campuses with Grubhub
  • How Ransomware Attacks Changed the Threat Landscape
  • Supply chain robotics company Symbotic reports $12 billion in orders
  • DHL becomes first to install Boston Dynamics’ Stretch robot to unload trucks
  • Five Reasons Home Solar is a No-Brainer
  • Leading Node.js frameworks for app development in 2023

Most Read

  • Top 20 electric vehicle charging station companies
    Top 20 electric vehicle charging station companies
  • DHL becomes first to install Boston Dynamics’ Stretch robot to unload trucks
    DHL becomes first to install Boston Dynamics’ Stretch robot to unload trucks
  • Why is My Car Key Stuck in the Ignition?
    Why is My Car Key Stuck in the Ignition?
  • Why Artificial Intelligence is Digital Marketing’s Future
    Why Artificial Intelligence is Digital Marketing’s Future
  • Difference Between Three-Phase and Single-Phase Power
    Difference Between Three-Phase and Single-Phase Power
  • Top 20 programmable logic controller manufacturers
    Top 20 programmable logic controller manufacturers
  • The Best Mechanical Engineering Design Software in 2022
    The Best Mechanical Engineering Design Software in 2022
  • Scientists have found more water in space than they ever knew possible
    Scientists have found more water in space than they ever knew possible
  • Supply chain robotics company Symbotic reports $12 billion in orders
    Supply chain robotics company Symbotic reports $12 billion in orders
  • Electric Future: Siemens providing software platform to support nascent electric airplane market
    Electric Future: Siemens providing software platform to support nascent electric airplane market

Overused words

ai applications automated automation automotive autonomous business china companies company control customers data design development digital electric global industrial industry logistics machine manufacturing market mobile operations platform process production robot robotic robotics robots safety software solution solutions system systems technologies technology time vehicle vehicles warehouse

Secondary Sidebar

Latest news

  • Ad Blockers Could Be Helpful for Your Privacy
  • LG robots deliver Korean food from Atlanta restaurant
  • Recycleye raises additional $17 million in Series A funding
  • 5 Reasons Why You Should Invest in Tesla this 2023
  • Starship expands robotic delivery to more college campuses with Grubhub
  • How Ransomware Attacks Changed the Threat Landscape
  • Supply chain robotics company Symbotic reports $12 billion in orders
  • DHL becomes first to install Boston Dynamics’ Stretch robot to unload trucks
  • Five Reasons Home Solar is a No-Brainer
  • Leading Node.js frameworks for app development in 2023

Footer

We are…

Robotics and Automation News was established in May, 2015, and is now one of the most widely-read websites in its category.

Please consider supporting us by becoming a paying subscriber, or through advertising and sponsorships, or by purchasing products and services through our shop – or a combination of all of the above.

Thank you.

Independent

Archivists

December 2022
M T W T F S S
 1234
567891011
12131415161718
19202122232425
262728293031  
« Nov   Jan »

Complex

Old-skool

This website and its associated magazine, and weekly newsletter, are all produced by a small team of experienced journalists and media professionals.

If you have any suggestions or comments, feel free to contact us at any of the email addresses on our contact page.

We’d be happy to hear from you, and will always reply as soon as possible.

Future-facing

Free, fair and legal

We support the principles of net neutrality and equal opportunities.

Member of The Internet Defense League

Copyright © 2023 · News Pro on Genesis Framework · WordPress · Log in

We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “Accept”, you consent to the use of ALL the cookies.
Do not sell my personal information.
Cookie SettingsAccept
Manage consent

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. These cookies ensure basic functionalities and security features of the website, anonymously.
CookieDurationDescription
cookielawinfo-checkbox-analytics11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checkbox-functional11 monthsThe cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checkbox-necessary11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-others11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-performance11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
viewed_cookie_policy11 monthsThe cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.
Functional
Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.
Performance
Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.
Analytics
Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.
Advertisement
Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. These cookies track visitors across websites and collect information to provide customized ads.
Others
Other uncategorized cookies are those that are being analyzed and have not been classified into a category as yet.
SAVE & ACCEPT