Robotics companies and “physical AI” labs have raised nearly $48 billion this year, according to PitchBook data reported by the Financial Times. A significant share of that money is going toward a resource that cannot simply be scraped from the web: examples of people doing physical work.
Language models can learn from huge stores of existing text and images. Robots need training material that connects perception with movement in varied real-world settings. That means recording people folding clothes, handling tools, moving objects and completing other tasks, then using those demonstrations to teach machines what to do next.
Inside the robot gyms
Companies including Generalist, Physical Intelligence, Neura and Skild are working with outside data providers or building facilities where people operate robotic arms through physical tasks. These “robot gyms” generate demonstrations that models can study and attempt to reproduce.
Other programs collect first-person video. The FT reported that startups are sending recording equipment to factories, offices, homes and vacation rentals around the world, gathering millions of hours of footage from a worker’s point of view. The approach is meant to capture the variety that a robot encounters outside a carefully controlled lab.
Physical Intelligence has tested robots in several San Francisco rental homes to see how its models handle different environments. The startup also plans to spend more than $1 billion over 12 months on services and computing power used to generate training data, according to the report.
Data companies that helped train language models are moving into the same market. Scale AI has opened a data facility in Mexico City, while Xdof operates facilities in Shenzhen and Jakarta. Xdof CEO Philipp Wu told the FT that the volume of data required makes comparable work in the United States uneconomical.
A costly bet on more data
The strategy rests on the hope that robotics will benefit from a version of the scaling pattern seen in language models: more training data produces better performance. Dyna said it observed improvements when training with as much as 1 million hours of first-person footage. But the case is not settled. UC Berkeley professor and Ambi Robotics co-founder Ken Goldberg cautioned that generating more data is no guarantee of a payoff.
The collection process also raises labor questions. The FT found factories in places including Gurugram, India, where workers were asked to wear recording equipment, sometimes without additional pay. Those recordings may become valuable training assets even when the people generating them receive no share of that value.
The funding surge shows how much investors and startups are willing to spend on the data bottleneck. Whether the result is a broadly capable generation of robots will depend on more than the number of recorded hours. The data must also capture useful tasks and enough variation for machines to work reliably outside the lab.
Comments