When the IEEE International Conference on Robotics and Automation (ICRA) first convened 40 years ago, the robotics community shared a clear vision: robots would one day combine elegant mathematical models with advanced computation to handle complex tasks.
Four decades later, the community is divided over how to reach that goal. That divide was on full display this May in Atlanta, where ICRA marked its anniversary with a unique closing keynote: a live Oxford-style debate on whether “data will solve robotics and automation”.
The debate, summarized and published this week in Science Robotics, brought together six leading roboticists, including three from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL):
- Daniela Rus, CSAIL director and the Andrew (1956) and Erna Viterbi Professor of Electrical Engineering and Computer Science;
- Russ Tedrake, the Toyota Professor at CSAIL, EECS, and the Department of Aeronautics and Astronautics; and
- Leslie Kaelbling, the Panasonic Professor of Computer Science and Engineering.
They were joined by international colleagues:
- Aude Billard of EPFL;
- Frank Park of Seoul National University; and
- Animesh Garg of Georgia Tech.
The debate was moderated by UC Berkeley’s Ken Goldberg, who framed the discussion with the question: “Will the future of robotics be written in code or in data?”
Rus and Tedrake argued that data-driven approaches, particularly those powered by large-scale machine learning, are critical to unlocking robots’ ability to function reliably in the real world.
“Physics gives us clean models for controlled environments, but the moment we step outside, those assumptions collapse,” Rus said. “Real-world tasks are unpredictable and human-centered. Robots need experience to adapt, and that comes from data.”
Rus’s “Distributed Robotics Lab” at CSAIL has embraced this, building multimodal datasets of humans performing everyday tasks, from cooking and pouring to handing off objects.
These recordings capture the subtleties of human action, from hand trajectories and joint torques to gaze and force interactions, providing a rich source of data for training AI systems. The goal is not just to have robots replicate actions, but to enable them to generalize across tasks and adapt when conditions change.
In the kitchen testbed at CSAIL, for example, Rus’s team equips volunteers with sensors while they chop vegetables, pour liquids, and assemble meals. The sensors record not only joint and muscle movements but also subtle cues such as eye gaze, fingertip pressure, and object interactions.
AI models trained on this data can then perform the same tasks on robots with precision and robustness, learning how to recover when ingredients slip or tools misalign. These real-world datasets let researchers capture “long-tail” scenarios – rare but critical occurrences that model-based programming alone would miss.
Tedrake discussed how scaling data transforms robot manipulation. His team has trained robots to perform dexterous tasks, such as slicing apples, observing diverse outcomes, and recovering when errors occur.
“Robots are now developing what looks like common sense for dexterous tasks,” he said. “It’s the same effect we’ve seen in language and vision: once you scale the data, surprising robustness emerges.”
In one example he showed, a bimanual robot equipped with simple grippers learned to core and slice apples. Each apple differed slightly in size, firmness, or shape, yet the robot adapted automatically, adjusting grip and slicing motions based on prior experience.
Tedrake explained that, as the demonstration dataset grew across multiple tasks, recovery behaviors – once manually programmed – began emerging naturally, a sign that data can encode subtle, high-level common-sense knowledge about physical interactions.
Kaelbling, who also spoke at the event, argued along with Billard and Park for the continuing importance of mathematical models, first principles, and theoretical understanding.
“Data can show us patterns, but models give us understanding,” Kaelbling said. “Without models, we risk systems that work, until they suddenly don’t. Safety-critical applications demand something deeper than trial-and-error learning.”
Billard argued that robotics differs fundamentally from vision or language: real-world data is scarce, simulations remain limited, and tasks involve infinite variability.
While large datasets have propelled progress in perception and natural language understanding, she cautioned that blindly scaling data without an underlying structure risks creating brittle systems.
Park emphasized the richness of inductive biases from physics and biology – principles of motion, force, compliance, and hierarchical control – that data-driven methods alone cannot fully capture.
He noted that carefully designed models can guide data collection and interpretation, helping ensure safety, efficiency, and robustness in complex tasks.
Garg, meanwhile, articulated the benefits of combining data-driven learning with structured models. He emphasized that while large datasets can reveal patterns and behaviors, models are necessary to generalize those insights and make them actionable.
“The best path forward may be a hybrid approach,” he said, “where we harness the scale of data while respecting the constraints and insights that models provide.”
Garg illustrated this with examples from collaborative manipulation tasks, where robots trained purely on raw data struggled with edge cases that a physics-informed model could anticipate.
The debate drew historical parallels. Humanity has often acquired “know-how” before “know-why”. From sailing ships and internal combustion engines to airplanes and early computers, engineers relied on empirical observation long before fully understanding the underlying scientific principles.
Rus and Tedrake argued that modern robotics is following a similar trajectory: data allows robots to acquire practical experience in messy, unpredictable environments, while models provide the structure necessary to interpret and generalize that experience.
This combination is essential, they said, to move from lab-bound experiments to robots capable of operating in homes, hospitals, and other real-world settings.
Throughout the debate, panelists emphasized the diversity of the robotics field itself. While deep learning has transformed perception and language tasks, robotics involves a multiplicity of challenges: high-dimensional control, variable human environments, interaction with deformable objects, and safety-critical constraints.
Tedrake noted that applying large pre-trained models from language directly to robots is insufficient; success requires multimodal learning and the integration of sensors that capture forces, motion, and tactile feedback.
Rus added that building large datasets across multiple robot platforms is crucial for generalization. “If we want robots to function across different homes, hospitals, or factories, we must capture the variety and unpredictability of the real world,” she said.
“Solving robotics is a long-term agenda,” Tedrake reflected. “It may take decades. But the debate itself is healthy. It means we’re testing our assumptions and sharpening our tools. The truth is, we’ll probably need both data and models – but which takes the lead, and when, remains unsettled.”
