KAIST's VOTP Teaches Robots Human Judgment From Just a Handful of Videos

Teaching a robot what "good" looks like has long been reinforcement learning's most expensive chore. To shape an agent's behavior toward human intentions, researchers have typically had to hand-label thousands to tens of thousands of action clips — this attempt was better than that one, this suture cleaner than that — a grind that has quietly throttled the rollout of physical AI. A team at the Korea Advanced Institute of Science and Technology (KAIST) says it can now get much of that judgment from just a handful of videos.

Their method, VOTP — short for Video-based Optimal Transport Preference — was accepted to the International Conference on Machine Learning (ICML) 2026 and selected for an Oral presentation, a distinction awarded to only 168 of the 23,918 submitted papers, roughly the top 0.7 percent. The conference runs this July at COEX in Seoul, and the VOTP oral was slated for July 8. The work comes out of the lab of Professor Chang D. Yoo in KAIST's School of Electrical Engineering, with PhD candidate Tung M. Luu as first author, alongside Hwanhee Kim and Younghwan Lee.

"The core of physical AI is making machines understand human intentions and choose the correct actions," Yoo said in KAIST's announcement. "Since VOTP can learn human judgment criteria with only a small number of videos, it is a core technology that will accelerate the era of robots making human-like judgments."

How VOTP works

The problem VOTP attacks sits at the heart of preference-based reinforcement learning, or PbRL — the same family of techniques behind RLHF, the reinforcement learning from human feedback that tuned modern chatbots. In PbRL, instead of hand-writing a reward function, you show a model pairs of behaviors and let humans pick the preferred one. The model infers a reward function from those preferences. It is elegant, but it is hungry: good reward functions have demanded enormous volumes of human comparisons, and that labeling cost is precisely what keeps preference learning from scaling to real robots.

VOTP's insight is to stop asking humans for most of those labels and let the machine label the rest itself. The team frames it as a semi-supervised problem. A human provides only a small set of preference labels — a few videos of good and bad examples. VOTP then leans on Video Foundation Models (ViFMs), large models pretrained to understand video, and uses their rich internal representation space as a shared coordinate system for comparing behaviors.

The clever mechanical piece is optimal transport, a mathematical tool for measuring how much "work" it takes to morph one distribution into another. VOTP uses optimal transport to align unlabeled video trajectories against the few labeled ones inside the ViFM's representation space. That alignment lets it generate high-fidelity pseudo-labels — machine-made preference judgments — for large amounts of unlabeled data. Those pseudo-labels then feed reward learning as though a human had produced them. In plain terms: teach the system your taste with a few examples, and it extrapolates that taste across a mountain of footage it has never been told anything about.

The design is deliberately modeled on how people pick up new tasks. As the KAIST team put it, they focused on "the way humans learn new tasks after seeing just a few demonstrations" — grasping a criterion from a handful of examples rather than exhaustive instruction.

The results

VOTP was tested across standard locomotion and manipulation benchmarks, the workhorses of RL research, and the paper reports that it outperforms existing offline PbRL methods under limited feedback budgets — the regime that matters when human labeling is the bottleneck. The team also stress-tested it against visual distractors, a common failure mode for vision-based agents, and validated it on real robotic tasks, where it learned useful reward signals from minimal human input rather than only in simulation.

Why It Matters

Feedback efficiency is the quiet constraint on almost everything in the "align a machine to human intent" business. RLHF made large language models usable, but it did so by throwing armies of human raters at comparison tasks. Extend that paradigm to robots — a surgical arm choosing how to suture, an autonomous vehicle threading a messy intersection, a humanoid deciding which of many motions best matches what a person actually wanted — and the labeling burden explodes, because physical behavior is higher-dimensional and harder to judge than text.

VOTP is a bet that video foundation models have absorbed enough general knowledge about how the physical world looks and moves that they can amortize most of that human judgment. If a few labeled videos can seed pseudo-labels good enough to train a reward function, the cost curve for teaching machines human preferences bends sharply downward. KAIST frames the technology as broadly applicable — robot arms, humanoids, autonomous vehicles, smart factories, drones, surgical robots, and even AI agents that operate computers directly. That last category is notable: the same feedback-efficiency problem dogs the digital agents now being trained to click through software, and a method that transfers preference learning from video could travel there too.

What to Watch

The paper's own hedges point to the open questions. Optimal-transport pseudo-labeling is only as good as the foundation model's representations; how VOTP holds up on tasks visually far from a ViFM's pretraining, and how gracefully it degrades when the initial human labels are noisy or contradictory, will determine how far it generalizes. The real-robot validation is encouraging but early, and the gap between benchmark locomotion and a production surgical or driving system remains wide. Still, if the top-0.7-percent ICML nod is any signal, the field sees feedback-efficient preference learning as one of the load-bearing problems for physical AI — and VOTP as a serious swing at it. Watch for follow-on work extending the pseudo-labeling trick to longer-horizon tasks and to computer-operating agents, where cheap human judgment is exactly what is missing.