Apple is investigating a more effective way to train humanoid robots by incorporating human instructors alongside robot demonstrators, a novel combined strategy the company has dubbed “PH2D”, according to a report on Apple Insider.
This development was detailed in a research paper published on Wednesday, just a week after the tech giant unveiled its Matrix3D and StreamBridge AI models, signaling a continued push into artificial intelligence and robotics.
The research paper, titled “Humanoid Policy ~ Human Policy”, addresses the shortcomings of conventional robot-training techniques and puts forward a new, scalable, and cost-effective solution.
Traditional methods that rely exclusively on robot demonstrators are often “labor-intensive” and necessitate “expensive teleoperated data collection”, according to the paper. Apple’s study proposes a combined approach that integrates human instructors into the training process to mitigate these issues.
A key aspect of this strategy is its cost-effectiveness, achieved by using modified consumer electronics to create training materials.
Specifically, an Apple Vision Pro was adapted to use only its lower-left camera for visual observation, while Apple’s ARKit was employed to capture 3D head and hand poses.
The company also utilized a modified Meta Quest headset equipped with ZED Mini Stereo cameras, offering a low-cost alternative for generating training data.
During the training process, human instructors, wearing these modified headsets, performed various hand manipulation tasks such as grasping and lifting objects, and pouring liquids.
Audible instructions were given as these actions were recorded, and the resulting footage was subsequently slowed down to be suitable for humanoid robot training.
To process this diverse training material from both human and robotic sources, Apple developed a model named the “Human-humanoid Action Transformer, Physical Human-Humanoid Data” – or PH2D.
The HAT model is designed to handle input from both human and robot demonstrators within a “generalizable policy framework”. Apple’s research indicates that this unique approach results in “improved generalization and robustness compared to the counterpart trained using only real-robot data”.
The study suggests that this combined training strategy offers significant advantages. Beyond its cost-effectiveness, robots trained using this method demonstrated better performance in certain tasks, such as vertical object grasping, when compared to those trained exclusively with robot demonstrators.
It is anticipated that Apple will likely implement this training methodology in its future robotics endeavors. While the company has previously showcased a robot-lamp prototype, reports suggest Apple is also developing a mobile robot for consumers capable of performing chores and simple tasks.
