Two weeks ago, OpenAI he said will restart its shuttered robotics program in 2021 — the latest sign that the biggest AI labs are racing to teach machines to function in the natural world. But building competent robots requires something the AI industry doesn’t yet have, which is training data that matches that used for language models.
This gap creates a new kind of infrastructure. Unlike LLMs trained on a vast sea of publicly available text, bots need data that captures physical interaction, and that kind of data barely exists. YouTube videos and footage taken by gig workers are low-fidelity and difficult to reconcile with the physical world.
XDOF (pronounced “ecks-doff”), emerging from stealth today, is betting that the next big bottleneck in AI isn’t models or brands, but the data feedback loop needed to teach robots how to interact with the natural world.
The startup aims to build the data pipelines, collection tools, and annotation systems that frontier labs and robotics companies can’t easily build—and has raised $70 million from Thrive Capital, Spark Capital, a16z, Lux, and WndrCo to do so. Co-founder and CEO Philipp Wu says XDOF, which has about 60 employees, already works with 20 clients, including several frontier AI labs, but he can’t name them.
“All the top labs are trying to pursue robotics,” Wu said. “We’ve already seen some of the negative consequences of being a little bit behind in the language model race … you don’t want to be in these kinds of situations where you’re pursuing this technology too late and everyone’s in this boat where natural AI is the next frontier.”
Wu faced this problem himself as a doctoral student at UC Berkeley. His focus was on enabling robots to learn skills from large-scale datasets. There was just one problem.
“We didn’t have large-scale data to work with,” he told TechCrunch. “There was this chicken-and-egg problem—first we needed to collect real data before we could even ask how to train a base model for robotics.”
Wu and future XDOF co-founder and CTO Fred Shentu worked on a project called GELLO, a low-cost teleoperation system that allows a human operator to control a robotic arm to generate training data. “It ended up being a very influential paper in robotics because a lot of people had similar needs and bottlenecks, and a lot of people started using this type of device to collect data,” Wu said.
Spotting the opportunity, Wu, Shentu and third co-founder and Chief Operating Officer Nemo Jin launched XDOF in October 2024 to provide a data ecosystem for companies pursuing robotics models. Mindful that providing data alone can be a dead-end business, the company also focuses on data cleaning, tool processing, and annotation — creating a self-reinforcing feedback loop for robot trainers.
As a starting point, the company is working with UC Berkeley’s AI Research Lab to release what it believes is the largest collection of high-quality robot training data ever assembled, called ABC. It includes 130,000 trajectories of robot handling data, 300 hours of simulation and 100 hours of evaluations. This kind of scaled pre-training data has never been available in academia.
“We’ve seen in language, imaging, and other areas that when models and data are released, the community achieves things that you wouldn’t necessarily expect,” David McAllister, a Ph.D. at Berkeley who helped organize the release, told TechCrunch.
The team has already used the data to train robots on benchmarking tasks like folding T-shirts and flattening boxes or loading AirPods into their cases.
Unlimited degrees of freedom
The company plans to work on three levels of a data pyramid. The most valuable layer is the teleoperation data collected for the actual robot being developed. Then come remote-controlled robots that collect more general data, such as with GELLO. and finally “self-centered” data collected from people performing everyday tasks, for which XDOF plans to build its own wearable sensors.
“Your choice of camera will affect the quality of your data—which will affect the performance of the hand-tracking algorithm,” Wu said. “If you don’t design the hardware well from the beginning, the data you collect can have very specific problems that you didn’t anticipate.”
The company plans to hire and train armies of remote operators and self-centered data operators around the world — a labor-intensive model that raises an obvious question: Why don’t the big labs do this data generation themselves?
“You need a warehouse of hundreds of thousands of square feet with hundreds of robots,” Wu said. “You have to maintain these robots, calibrate their physical parameters, and properly train the operators.”
This is a build that requires focus, capital and operational scale that most AI labs would prefer to outsource – which is exactly what XDOF is betting on.
The name XDOF is a play on the robotics term “degrees of freedom,” which describes the number of independent movements a robot can perform. Your arm, from shoulder to wrist, has seven degrees of freedom. Humanoid robotics company Figure AI has 30. The X in the company’s name captures its ambition: “Arbitrary degrees of freedom, unlimited degrees of freedom,” says Wu.
When you purchase through links in our articles, we may earn a small commission. This does not affect our editorial independence.
