Collecting robot training data is a dirty and unattractive business. Some AI labs are already paying XDOF to do this

Two weeks ago, OpenAI He said It will relaunch a robotics program it shuttered in 2021 — the latest sign that the largest AI labs are racing to teach machines how to work in the physical world. But building capable robots requires something the AI ​​industry doesn’t yet have: training data that matches that used in language models.

This gap creates a new type of infrastructure work. Unlike LLM students who are trained on a vast sea of ​​publicly available text, robots need data that captures physical interaction, and this kind of data rarely exists. The videos and footage captured by gig workers on YouTube are low-resolution and difficult to reconcile with the physical world.

XDOF (pronounced “ecks-doff”), which emerges from stealth today, is betting that the next big bottleneck in AI is not models or chips, but the feedback loop of data needed to teach robots how to interact with the physical world.

The startup aims to build data pipelines, collection tools, and annotation systems that frontier labs and robotics companies can’t easily build themselves — and has raised $70 million from Thrive Capital, Spark Capital, a16z, Lux, and WndrCo to do just that. XDOF, which has about 60 employees, already works with 20 clients including several frontier AI labs, but can’t name them, says co-founder and CEO Philip Wu.

“All major laboratories are trying to pursue robotics development,” Wu said. “We’ve already seen some disadvantages of being a little bit behind in the language model race… You don’t want to be in that kind of situation where you’re chasing this technology too late, and everyone is in this boat where physical AI is the next frontier.”

Wu encountered this problem himself when he was a doctoral student at the University of California, Berkeley. His focus was on enabling robots to learn skills from large-scale data sets. There was only one problem.

“We didn’t have large-scale data to work with,” he told TechCrunch. “There was a chicken-and-egg problem, in that we first needed to actually collect the data before we could even ask how to train a basic robotics model.”

Wu and future XDOF founder and CTO, Fred Shentu, worked on a project called GELLO, a low-cost teleoperation system that allows a human operator to control a robotic arm to generate training data. “It ended up being a very influential paper in robotics, because a lot of people had similar needs and bottlenecks, and many started taking advantage of this type of device to collect data,” Wu said.

Wu, Shintu, and third co-founder and COO Nimmo Jin saw the opportunity, and launched XDOF in October 2024 to provide a data ecosystem for companies pursuing robotics models. Realizing that providing data alone can be a dead-end job, the company is also focusing on data cleaning, tools, and annotations — creating a self-reinforcing feedback loop for bot coaches.

As a starting point, the company is collaborating with UC Berkeley’s Artificial Intelligence Research Lab to release what it believes is the largest set of high-quality robot training data ever compiled, dubbed ABC. It includes 130,000 robot processing data streams, 300 hours of simulation, and 100 hours of evaluations. This type of expanded pre-training data has not been available to academia before.

“We’ve seen in language, image generation, and other fields that when models and data are released, the community achieves things you didn’t necessarily expect,” David McAllister, a PhD student at Berkeley who helped organize the release, told TechCrunch.

The team has already used the data to train robots on standard tasks like folding shirts, leveling boxes, or loading AirPods into their cases.

Unlimited degrees of freedom

The company plans to work across three levels of the data pyramid. The most valuable level is the teleoperation data collected on the actual robot being deployed; Next come remote-controlled robots that collect more general data, as is the case with GELLO; And finally, “egocentric” data collected by humans performing everyday tasks, for which XDOF plans to build its own wearable sensors.

“The choice of your camera will affect the quality of your data, which will affect how the hand tracking algorithm performs,” Wu said. “If you don’t design the devices well from the beginning, the data you collect may have very specific problems that you didn’t expect.”

The company plans to hire and train armies of data operators and selfish data operators around the world — a labor-intensive model that raises an obvious question: Why don’t big labs produce the data themselves?

“You need a warehouse of hundreds of thousands of square feet that houses hundreds of robots,” Wu said. “You need to maintain these robots, calibrate their physics, and properly train the operators.”

It’s a construction process that requires the focus, capital, and operational scale that most AI labs prefer to outsource — which is exactly the market XDOF is betting on.

The name XDOF is a play on the robotics term “degrees of freedom,” which describes the number of independent movements a robot can make. Your arm, from shoulder to wrist, has… Seven degrees of freedom. The humanoid robotics company’s latest robot contains 30. The X in the company’s name symbolizes its ambition: “arbitrary degrees of freedom, infinite degrees of freedom,” says Wu.