The robotics industry is enjoying a surge of investment, media attention, and ambitious promises about the future of humanoid machines.
Companies are announcing plans to manufacture thousands of robots, while advances in artificial intelligence have fueled expectations that general-purpose robots may soon become commonplace in factories, warehouses, workplaces, and even homes.
Yet beneath the excitement lies a fundamental technical challenge that many researchers believe remains far from solved.
While robots have become increasingly capable of moving through environments, reliably interacting with the physical world is a much harder problem.
Walking across a room, opening a door, folding laundry, organizing clutter, or handling unfamiliar objects all require a level of manipulation and environmental understanding that continues to challenge even the most advanced robotic systems.
Yunzhu Li has spent much of his career working on exactly that problem. An assistant professor at Columbia University and co-founder of SceniX, Li specializes in robotics, simulation, physical AI, and robot learning.
His research focuses on helping machines understand and interact with the real world by combining simulation, artificial intelligence, and large-scale data generation.
SceniX is developing tools that enable robotics companies to generate training data, build realistic simulation environments, evaluate robot performance, and accelerate the transition from laboratory demonstrations to reliable real-world deployment.
The company’s approach is based on a simple premise: robots themselves are relatively easy to model, but the environments they operate in are not. Closing that gap may prove critical to the next stage of robotics development.
In this Q&A, Li discusses why manipulation remains one of the biggest bottlenecks in robotics, the limitations of today’s humanoid robots, the growing importance of simulation in physical AI, and where he believes industry expectations may be running ahead of reality.
He also explains why warehouses, factories, laboratories, and retail environments are likely to see widespread robotic deployment before ordinary homes, and what still needs to happen before robots can operate reliably in truly unstructured environments.
Interview with Yunzhu Li
Robotics & Automation News: There has been enormous attention around humanoid robots recently, particularly around manufacturing scale and deployment targets. Do you think the industry is becoming overly focused on locomotion rather than real-world manipulation capability?
Yunzhu Li: I would not say the industry is ignoring manipulation. People understand that manipulation is critical for unlocking the real commercial potential of humanoid robots.
We see so much attention around locomotion partly because it is much closer to becoming “solved,” at least to the point where robots can produce compelling public demos.
Manipulation is still at a different level of difficulty. Walking across a room mostly requires the robot to model and control its own body.
Manipulation requires the robot to understand objects, materials, geometry, contact, and how the environment changes through interaction.
Handling deformable objects, organizing clutter, or reliably manipulating unfamiliar items in new environments remains much harder.
So the focus on locomotion reflects where the technology is more mature and visible. But the larger bottleneck for broad commercial deployment is still general-purpose manipulation.
R&AN: You’ve argued that walking across a room is a very different problem from reliably interacting with unpredictable objects and environments. Why is robotic manipulation still such a difficult challenge for the industry to solve?
YL: The key difference is that locomotion is mostly about controlling the state of the robot, while manipulation is about precisely changing the state of the environment.
If a robot kicks a rock while walking, it may be fine as long as the robot stays stable. But if the task is to manipulate that rock, the robot needs to know exactly where it moves, how it rotates, and whether it ends up in the desired state.
That is what makes manipulation difficult. The robot has to reason about objects, materials, geometry, contact, friction, and uncertainty, not just its own body.
Small errors can quickly change the outcome of the task, especially with deformable objects, cluttered scenes, articulated objects, or unfamiliar items.
R&AN: Many humanoid demonstrations still take place in highly controlled environments. What are the biggest technical barriers preventing robots from operating reliably in ordinary homes, warehouses, or workplaces?
YL: Controlled environments reduce uncertainty. In ordinary homes, warehouses, or workplaces, robots have to handle clutter, lighting changes, moving people, changing object locations, and many edge cases that are hard to anticipate.
A major barrier is that robots still do not have a reliable understanding of how the environment changes through interaction. Humans continuously build mental models of the world and update them as we act. Robots are much less reliable at doing this in real time, especially when contact changes the scene.
Safety and long-term reliability are also major challenges. A small perception or planning error can quickly become a deployment issue outside controlled settings.
R&AN: SceniX focuses heavily on simulation and synthetic environments for robotics training. Why is simulation becoming so important for developing physical AI systems?
YL: Simulation has been central to most successful robotics and automation systems. If you look at planes, rockets, drones, Roombas, robot dogs, or recent progress in humanoid locomotion, a big reason these systems work is that simulation works.
The simulation does not have to be perfect. Even approximate models can go a long way if they capture the right structure of the problem.
For physical AI, simulation is especially important because collecting real-world robot data is slow, expensive, and difficult to scale safely. In simulation, we can generate diverse training data, test edge cases, and evaluate systems before deploying them in the real world.
At SceniX, we focus on building digital environments from real-world inputs, so robots can learn not only about their own bodies, but also about the environments they need to interact with.
R&AN: One issue you highlighted is that robots themselves are relatively easy to model, but the real world is not. Could advances in simulation eventually narrow that gap enough for robots to generalize effectively outside controlled settings?
YL: Yes, I think advances in simulation can narrow that gap significantly, but only if we move beyond modeling the robot itself. A robot is relatively well defined, especially when we designed and manufactured it ourselves. We know its geometry, joints, sensors, actuators, and control limits.
The real world is much harder because it contains diverse objects, materials, contact dynamics, deformable structures, and constant changes.
The key is not necessarily to make simulation perfectly realistic in every detail. The goal is to capture the aspects of the world that matter for robot decisions and physical interaction.
If we can build simulations that are grounded in real-world observations, and generate enough useful variation around those observations, then simulation can become a bridge between limited real-world data and robust deployment outside controlled settings.
R&AN: There is currently a huge amount of investment flowing into humanoid robotics. From your perspective, where do you think expectations are realistic, and where might the industry be overestimating near-term progress?
YL: Expectations around hardware, manufacturing, and locomotion are relatively realistic. We will continue to see humanoids become more capable, reliable, and easier to produce at scale.
Where I think expectations may be too aggressive is general-purpose manipulation in fully unstructured environments. I expect faster progress first in semi-structured settings, such as warehouses, factories, labs, or retail backrooms, where the environment can be partially standardized and the task distribution is clearer.
Ordinary homes are much harder. Robots there need to handle messy rooms, unfamiliar objects, deformable items, and constantly changing layouts. I am optimistic, but broad deployment in open-ended environments will take longer than many current projections suggest.
R&AN: SceniX says its platform supports data generation, training, evaluation, and predictive monitoring for robotics systems. How does that approach help robotics companies move from impressive demos toward reliable real-world deployment?
YL: The main challenge in moving from demos to deployment is iteration speed. For language and vision models, we have internet-scale data and can improve models largely in the virtual world.
For robotics, we do not have internet-scale robot data, and every real-world experiment is slow, expensive, and limited by hardware, safety, and environment setup.
SceniX is trying to make that iteration loop much faster for robot policies through a real-to-sim-to-real approach. By building simulation environments with strong sim-real alignment, we can generate data, train policies, evaluate them across many conditions, and identify likely failure modes before deployment.
This is especially important for semi-structured and unstructured environments, where a policy that works in one demo still needs to be tested against variations in objects, layouts, contacts, and edge cases.
The goal is to help robotics companies move faster from a promising demo to a reliable product, reducing the number of costly real-world iterations needed before deployment.
R&AN: As both a researcher and startup co-founder, how do you balance long-term scientific challenges with the commercial pressure to deliver practical robotics solutions quickly?
YL: I see them as closely connected. The long-term scientific challenge is to build robots that can understand and interact with the physical world. The commercial challenge is to identify where today’s technology can already generate value.
In academia, we often study the most general version of a problem. In a startup, we can focus more on execution: defining the operational domain, measuring success clearly, and delivering practical solutions to customers. I feel fortunate to work with extremely talented teams in both settings, which helps connect long-term research with real-world needs.
My goal is to work on problems that are technically deep, grounded in real needs, and useful for building more general-purpose robotic intelligence over time.
R&AN: Looking ahead five to 10 years, what kinds of robotic manipulation tasks do you believe will finally become commercially reliable – and which problems remain much farther away than people currently assume?
YL: The key factor is variation. The more variation there is in the objects, layouts, materials, and task goals, the harder the manipulation problem becomes.
Over the next five to 10 years, I expect robotic manipulation to become commercially reliable first in settings where variation is limited or can be controlled. Examples include warehouse logistics, industrial handling, lab automation, retail backrooms, and some repetitive workplace tasks.
These environments still have complexity, but the task distribution is clearer and the environment can often be partially standardized or digitized.
What remains much harder is general-purpose manipulation in fully unstructured environments, such as ordinary homes. Robots there need to handle messy rooms, unfamiliar objects, deformable items, changing layouts, and long-horizon tasks where small errors can accumulate.
I think we will see many useful robotic systems in the next decade, but human-level adaptability in open-ended environments is still much farther away.
Main image: A robotic simulation using Nvidia’s Isaac sim. Courtesy of Nvidia
