Runway, one of the best-known developers of generative AI video technology, is moving into robotics with Praxis-1, an open-weight AI model designed to use knowledge learned from video to control physical robots.
“Open-weight” means developers can download the model and run or adapt it themselves, rather than only accessing it through Runway’s servers.
The company describes Praxis-1 as its first “world action model” and says it is based on the same large-scale video pretraining underlying its general world models.
Runway is currently testing Praxis-1 with robotics companies Noble Machines, Standard Bots and Ultra Robotics on several types of robot hardware. It plans to make the model publicly available in the coming months.
The move takes Runway beyond its established generative video business and into the rapidly developing physical AI and robotics market.
Runway says: “Today we’re announcing Praxis-1, our first open-weight world action model. It’s built on the same large-scale video pretraining behind our general world models.
“We’re actively testing Praxis-1 with early partners across a variety of embodiments, and will release it publicly in the coming months.”
Using internet video to train robots
The central idea behind Praxis-1 is that robots do not necessarily need to learn predominantly from enormous quantities of robot-generated training data.
Collecting real-world robotics data is expensive and time-consuming, particularly when it requires robots to be teleoperated through tasks repeatedly.
Runway instead wants to exploit the vastly larger quantities of ordinary video available for training AI models.
Video can teach a model about object behavior, movement, physical interactions and how humans perform tasks. Runway’s argument is that a robot policy beginning with this understanding of the physical world has an advantage over one trained primarily from robot action data.
The company says: “A policy that already understands physical plausibility and object behavior from video pretraining has an enormous head start on one built from action data alone.”
Runway compares the approach with the development of large language models, which acquire a broad understanding of language from large quantities of text before being adapted for more specific tasks.
The company says its experiments indicate a similar scaling effect in robotics, with robot policy performance improving as the amount of third-person video used during training increases.
Web video approaches robot training data
One experiment presented by Runway compares policies pretrained using ordinary web video with those pretrained using teleoperated robot video.
After fine-tuning, the company reports a final placement error of 16.1 cm for the model initially trained on web video, compared with 16.0 cm for the model trained using teleoperated robot footage.
Runway notes that the difference is not statistically significant within the reported uncertainty.
The company is also testing Praxis-1 on more difficult manipulation problems involving cluttered environments, transparent objects, deformable materials and groups of similar-looking objects.
In another demonstration, Runway shows the same policy operating in both a controlled studio environment and a domestic kitchen without retraining.
Runway brings world models into robotics
Praxis-1 builds on Runway’s development of interactive video and world models including Solaris and GWM Worlds 2.
These models attempt to represent how environments and objects change over time rather than simply generating individual images or predetermined video sequences.
Runway says: “By teaching our models how to generate accurate physics – how objects behave, how hands move, what a task looks like partway through – we’ve created dynamic, complex environments for agent training in the digital and physical world.”
Praxis-1 applies that approach to robotics, with Runway aiming to develop a general-purpose policy capable of working across different robot embodiments and environments.
The company also says previous experiments found that simulating robot policies inside its world model produced results with a 0.95 correlation to subsequent real-world performance.
Testing on three different robot platforms
Praxis-1 is being tested with three early robotics partners.
Noble Machines is evaluating the model for bimanual manipulation, while Standard Bots is running it on its RO1 six-degree-of-freedom robotic arm. Ultra is testing Praxis-1 with a mobile robotic platform.
Runway says the early-access program will be expanded to additional companies ahead of the model’s public release.
The testing program is intended to evaluate both performance and safety across different robot configurations and operating environments.
Praxis-1 will be released with open weights
Runway also plans to make Praxis-1 available with open weights rather than restricting access to a proprietary hosted model.
The company links that decision partly to its view that greater interoperability between AI models and robot hardware will be important as physical AI develops.
Runway says: “We view open world models as a compounding advantage that gives hardware developers flexibility and control they don’t currently have.”
The company is currently accepting applications from robotics developers interested in testing Praxis-1 on their hardware before its wider release.
The launch puts Runway into an increasingly important area of AI research: using large-scale world models and knowledge acquired from video to train machines capable of interacting with the physical world.
If the approach scales, the significance of Praxis-1 may lie in reducing one of robotics’ persistent bottlenecks – obtaining enough diverse real-world training data to produce robots that can operate reliably outside tightly controlled environments.
