• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to secondary sidebar
  • Skip to footer
  • Home
  • Subscribe
  • Your Membership
    • Edit Your Profile
  • Services
    • Advertising
    • Case studies
    • Design
    • Email marketing
    • Lead generation
    • Magazine
    • Press releases
    • Publishing
    • Sponsored posts
    • Webcasting
    • Webinars
    • White papers
    • Writing
  • Shop
    • My Account
    • Cart
  • About
    • Contact
    • Privacy
    • Terms of use
  • Events

Robotics & Automation News

Market trends and business perspectives

  • News
  • Features
  • Video
  • Webinars
  • White papers
  • Press releases
  • Featured companies
    • BlueBotics
    • SICK Sensor Intelligence
    • Vicor Power
nvidia ceo jensen huang

Nvidia opens robotics research lab in Seattle

January 14, 2019 by Abdul Montaqim

Nvidia is opening a new robotics research lab in Seattle near the University of Washington campus led by Professor Dieter Fox, senior director of robotics research at Nvidia and professor in the UW Paul G. Allen School of Computer Science and Engineering. (See video below.)

The charter of the lab is to drive breakthrough robotics research to enable the next generation of robots that perform complex manipulation tasks to safely work alongside humans and transform industries such as manufacturing, logistics, healthcare, and more.

Professor Dieter Fox says: “In the past, robotics research has focused on small, independent projects rather than fully integrated systems.

“We’re bringing together a collaborative, interdisciplinary team of experts in robot control and perception, computer vision, human-robot interaction, and deep learning.” 

Close to 50 research scientists, faculty visitors, and student interns will perform foundational research in these areas. To ensure the research stays relevant to real-world robotics problems, the lab will investigate its work in the context of large scale, realistic scenarios for interactive manipulation.

researchers SEA-Robotics-38 copy
Researchers at the Nvidia Seattle robotics lab

What’s cooking in the robotics research lab?

The first of these challenge scenarios is a real-life kitchen where a mobile manipulator solves a variety of tasks, ranging from retrieving objects from cabinets to learning how to clean the dining table to help a person cook a meal.

At an Open House event on January 11, the Seattle lab demonstrated its first manipulation system in their kitchen. The mobile manipulator integrates state-of-the-art techniques to detect and track objects, keep track of the state of doors and drawers in the kitchen, and open/close them to get access to objects for manipulation.

An experimental robot to research and analyse some of the tasks in the kitchen.
An experimental robot to research and analyse some of the tasks in the kitchen.

These approaches can be applied in arbitrary environments, only requiring 3D models of relevant objects and cabinets.

Building on Nvidia’s expertise in physics-based, photorealistic simulation, the robot uses deep learning to detect specific objects solely based on its own simulation, not requiring any tedious manual data labeling.

Nvidia’s highly parallelized GPU processing enables the robot to keep track of its environment in real-time, using sensor feedback for accurate manipulation and to quickly adapt to changes in the environment.

The robot uses the Nvidia Jetson platform for navigation and performs real-time inference for processing and manipulation on Nvidia Titan GPUs. The deep learning-based perception system was trained using the cuDNN-accelerated PyTorch deep learning framework.

What makes the system unique is that it integrates a suite of cutting-edge technologies developed by the lab researchers. These technologies working together enable detection of objects, tracking the position of doors and drawers, and generate control commands so that the robot can grasp and move objects from one place to another.

The system is comprised and built on the following technologies:

  • Dense Articulated Real-Time Tracking (DART): DART, which was first developed in Fox’s UW robotics lab, uses depth cameras to keep track of a robot’s environment. It is a general framework for tracking rigid objects, such as coffee mugs and cereal boxes, and articulated objects often encountered in indoor environments, like furniture and tools, as well as human and robot bodies including hands and manipulators.
  • Pose-CNN: 6D Object Pose Estimation: Detecting the 6D pose and orientation of known objects is a crucial capability for robots that pick up and move objects in an environment. This problem is challenging due to changing lighting conditions and complex scenes caused by clutter and occlusions between objects. Pose-CNNisa deep neural network trained to detect objects using regular cameras.
  • Riemannian Motion Policies (RMPs) for Reactive Manipulator Control: RMPs are a new mathematical framework that consistently combines a library of simple actions into complex behavior. RMPs allow the team to efficiently program fast, reactive controllers that use the detection and tracking information from Pose -CNN and DART to safely interact with objects and humans in dynamic environments.
  • Physics-based Photorealistic Simulation: Nvidia’s Isaac Sim tool enables the generation of realistic simulation environments that model the visual properties of objects as well as the forces and contacts between objects and manipulators. A simulated version of the kitchen is used to test the manipulation system and train the object detection network underlying Pose-CNN. If done on a real robot this training and development process would be an expensive and time-consuming process. Once simulation models of objects and the environment are available, training and testing can be done far more efficiently, saving precious development time.

Fox says: “We really feel that the time is right to develop the next generation of robots. By pulling together recent advances in perception, control, learning, and simulation, we can help the research community solve some of the world’s greatest challenges.”

(Main picture: Jensen Huang, CEO of Nvidia, pictured at the new Seattle robotics lab.)

Print Friendly, PDF & Email

Share this:

  • Print
  • Facebook
  • LinkedIn
  • Reddit
  • Twitter
  • Tumblr
  • Pinterest
  • Skype
  • WhatsApp
  • Telegram
  • Pocket

You might also like…

Filed Under: Computing, Features Tagged With: nvidia, research, robotics, seattle

Join the Robotics & Automation News community

Primary Sidebar

Latest articles

  • Wikifactory secures $2.5 million funding to expand its manufacturing marketplace
  • Motional and Uber launch autonomous deliveries of food
  • Commercial vehicle maker MAN selects Magazino robot for warehouse work
  • Continental to unveil new autonomous mobile robots for first time at LogiMAT
  • GreyOrange raises $110 million for ‘growth financing’
  • Ally Robotics aims to raise $8.5 million to bring automation to the restaurant industry
  • ABB launches new robotic depalletizer to ‘reduce complexity and improve efficiency’
  • Switzerland’s largest trade fair celebrates its 20th anniversary with a promising edition
  • Seegrid launches online knowledge base for its autonomous mobile robots
  • Hydroplane highlights its hydrogen fuel cell powerplant development

Most Read

  • Track your lost Android
    Track your lost Android
  • Top 20 electric vehicle charging station companies
    Top 20 electric vehicle charging station companies
  • Difference Between Three-Phase and Single-Phase Power
    Difference Between Three-Phase and Single-Phase Power
  • Top 25 vertical farming companies
    Top 25 vertical farming companies
  • Scientists have found more water in space than they ever knew possible
    Scientists have found more water in space than they ever knew possible
  • The Best Mechanical Engineering Design Software in 2022
    The Best Mechanical Engineering Design Software in 2022
  • ABB launches new robotic depalletizer to ‘reduce complexity and improve efficiency’
    ABB launches new robotic depalletizer to ‘reduce complexity and improve efficiency’
  • Top 20 programmable logic controller manufacturers
    Top 20 programmable logic controller manufacturers
  • Ally Robotics aims to raise $8.5 million to bring automation to the restaurant industry
    Ally Robotics aims to raise $8.5 million to bring automation to the restaurant industry
  • Hydroplane highlights its hydrogen fuel cell powerplant development
    Hydroplane highlights its hydrogen fuel cell powerplant development

Overused words

abb ai applications automated automation automotive autonomous business china companies company control customers data design development digital electric global industrial industry logistics machine manufacturing market mobile platform process production robot robotic robotics robots safety software solution solutions system systems technologies technology time vehicle vehicles warehouse

Secondary Sidebar

Latest news

  • Wikifactory secures $2.5 million funding to expand its manufacturing marketplace
  • Motional and Uber launch autonomous deliveries of food
  • Commercial vehicle maker MAN selects Magazino robot for warehouse work
  • Continental to unveil new autonomous mobile robots for first time at LogiMAT
  • GreyOrange raises $110 million for ‘growth financing’
  • Ally Robotics aims to raise $8.5 million to bring automation to the restaurant industry
  • ABB launches new robotic depalletizer to ‘reduce complexity and improve efficiency’
  • Switzerland’s largest trade fair celebrates its 20th anniversary with a promising edition
  • Seegrid launches online knowledge base for its autonomous mobile robots
  • Hydroplane highlights its hydrogen fuel cell powerplant development

Footer

We are…

Robotics and Automation News was established in May, 2015, and is now one of the most widely-read websites in its category.

Please consider supporting us by becoming a paying subscriber, or through advertising and sponsorships, or by purchasing products and services through our shop – or a combination of all of the above.

Thank you.

Independent

Archivists

January 2019
M T W T F S S
 123456
78910111213
14151617181920
21222324252627
28293031  
« Dec   Feb »

Complex

Old-skool

This website and its associated magazine, and weekly newsletter, are all produced by a small team of experienced journalists and media professionals.

If you have any suggestions or comments, feel free to contact us at any of the email addresses on our contact page.

We’d be happy to hear from you, and will always reply as soon as possible.

Future-facing

Free, fair and legal

We support the principles of net neutrality and equal opportunities.

Member of The Internet Defense League

Copyright © 2022 · News Pro on Genesis Framework · WordPress · Log in

We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “Accept”, you consent to the use of ALL the cookies.
Do not sell my personal information.
Cookie SettingsAccept
Manage consent

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. These cookies ensure basic functionalities and security features of the website, anonymously.
CookieDurationDescription
cookielawinfo-checkbox-analytics11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checkbox-functional11 monthsThe cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checkbox-necessary11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-others11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-performance11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
viewed_cookie_policy11 monthsThe cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.
Functional
Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.
Performance
Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.
Analytics
Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.
Advertisement
Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. These cookies track visitors across websites and collect information to provide customized ads.
Others
Other uncategorized cookies are those that are being analyzed and have not been classified into a category as yet.
SAVE & ACCEPT