Training robots to perform varied physical tasks requires large amounts of data, but collecting that data through teleoperation is expensive and time-consuming. Three recent studies propose different ways to address this constraint: converting human videos into robot training material, using shared exoskeleton hardware to improve demonstrations and creating a large physical benchmark that allows experience to be reused across tasks, environments and robot platforms.
Robot learning has traditionally depended on people remotely controlling physical robots while their movements, camera images and sensor readings are recorded. This produces data that closely matches the robot on which the policy will operate, but the process requires hardware, trained operators and repeated preparation of the workspace.
The problem becomes more pronounced as developers attempt to train general-purpose robot models. A system expected to handle different objects, environments and unexpected situations needs considerably more varied experience than a robot programmed for one fixed industrial task.
Recent research from HuRo, SEED-UMI and FolDeX illustrates three emerging approaches to this problem. Rather than relying exclusively on conventional robot teleoperation, the projects attempt to use existing human video, improve the correspondence between human and robot demonstrations, or combine experience collected across multiple systems.
Human videos converted into robot data
HuRo, a project accepted at the Conference on Robot Learning 2026, focuses on converting human-action videos into data suitable for pretraining vision-language-action models. Human video is widely available, but a robot cannot directly reproduce the recorded movements. Human hands and arms differ from robotic mechanisms, while the original footage normally lacks robot joint positions and action commands.
The researchers developed a processing pipeline that converts heterogeneous human videos into robot-aligned observations and action trajectories. It also estimates intermediate information that is missing from the original footage.
Using videos from five sources, the team created approximately 630,000 converted episodes containing 142 million processed frames. The resulting dataset was used to pretrain robot policies before testing them on four physical manipulation tasks.
According to the reported results, increasing the amount of converted video used during pretraining raised overall task completion from 51.5% to 80.3%. Under changes in object position and visual appearance, completion increased from 34.9% to 72.2%. The experiments also indicated that converting both the visual observations and the human actions produced better results than transferring visual information alone.
The scale of the dataset is notable, but the physical evaluation remains limited to four tasks. HuRo therefore does not demonstrate that a robot can watch an arbitrary online video and learn the activity it depicts. The conversion process still depends on models that interpret human movement and translate it into a form compatible with robot hardware.
Shared hardware reduces the embodiment gap
SEED-UMI takes a more hardware-oriented approach. Instead of translating freely recorded human motion to a robotic hand, the system has the human demonstrator and robot use the same hand exoskeleton.
The exoskeleton’s joint encoders record a common physical representation of movement. Wrist-mounted cameras also observe the same outer mechanism during human demonstrations and subsequent robot operation. The researchers say this reduces discrepancies between the images and joint measurements collected from the two embodiments.
This approach is intended particularly for contact-rich manipulation, where minor differences in finger position and applied motion can determine whether an action succeeds. Conventional exoskeleton systems generally record movement on the human side and then use an open-loop mapping to translate it to the robot. That mapping can become less accurate once the hand touches or constrains an object.
Across five tasks, SEED-UMI reportedly collected demonstrations three times more efficiently than exoskeleton-based robot teleoperation. Policies trained with the data achieved an average physical rollout success rate of 70%.
The method improves alignment by introducing specialised hardware, however. Both demonstrator and robot must use the same mechanism, which could make the approach more difficult to transfer to hands with substantially different designs. A 70% average success rate also means that nearly one in three evaluated attempts remained unsuccessful.
SEED-UMI is currently a preprint and has not yet been peer reviewed.
Reusing experience across robots and tasks
While HuRo and SEED-UMI address data collection, FolDeX focuses on evaluating whether robot experience can be reused. The physical-world benchmark contains more than 2,000 hours of real-robot data covering over 20 tasks and more than ten robot embodiments.
Its main application is garment folding, a difficult test for robot learning because fabric continuously changes shape and can be partially hidden during manipulation. Completing a folding task also requires a sequence of coordinated actions rather than a single grasp-and-place movement.
FolDeX is organised around several forms of data transfer. These include using human interventions and recovery attempts collected during deployment, transferring experience between garment categories, reusing data after changes in lighting or workspace layout and applying experience from one robot embodiment to another. The dataset also includes transfers between rigid-object and deformable-object manipulation.
The project includes a platform through which external researchers can submit policies for physical evaluation. Tests use standardised initial conditions and objects withheld from the training material. This could provide a more comparable measure of real-world robot performance than results produced independently in different laboratories.
However, FolDeX primarily introduces a dataset and evaluation infrastructure. It does not establish that current models can already transfer reliably across all included tasks and robot types. The total number of recorded hours also does not by itself indicate how diverse the data is or how evenly it is distributed across the different platforms.
Like SEED-UMI, FolDeX is a preprint and has not yet been peer reviewed.
Multiple routes beyond conventional teleoperation
The three projects do not eliminate the need for physical robot data. HuRo still requires real-world testing and robot-compatible conversion, while SEED-UMI depends on a shared physical interface. FolDeX itself was assembled through extensive real-robot operation.
They do, however, show how robot-learning research is broadening its approach to data. Human videos could provide scale, shared hardware could improve the accuracy of contact-rich demonstrations, and multi-platform benchmarks could help determine which forms of experience transfer between robots and environments.
The underlying issue is no longer simply how many hours of training data can be collected. Developers must also determine whether that data represents the states a robot will encounter, whether human movement can be translated accurately to a different body and whether experience collected on one machine remains useful on another.
Sources
HuRo: Robotizing Human Videos for Scalable VLA Pretraining
SEED-UMI: Sharing the Exoskeleton between Human and Robot for One-to-One Dexterous Demonstration
FolDeX: A Physical-World Benchmark for Long-Horizon Robotic Manipulation of Deformable Objects
