LEGS Simulator   Real Robot

LEGS Fine-Tuning Teleop-Free VLAs for Humanoid Loco-manipulation in an Embodied Gaussian Splatting World

Hojune Kim, Timothy Chen, Jiankai Sun, Lars W. Osterberg,
Qianzhong Chen, Ke Wang, Mac Schwager
Stanford University
CoRL 2026
arXiv Paper Code
0 teleoperation demos needed
1,230 real-robot trials on a Unitree G1
9/9 experiments match or beat human teleop
>15× cheaper adaptation to new scenes & objects

Three pick-and-place tasks of increasing whole-body difficulty × three VLA backbones (ψ0, π0.5, GR00T N1.6).

Collecting teleoperation demonstrations for humanoids is slow and expensive — and simulator-trained VLA policies have so far transferred poorly to real humanoid loco-manipulation. What if a phone scan is all you need? LEGS composites the robot and objects over a photorealistic 3D Gaussian Splatting background inside MuJoCo, and procedurally generates labeled demonstrations — no teleoperation, no seed demos, no human video. We find that LEGS:

  • Matches or beats human teleoperation on all nine (backbone, task) experiments — zero-shot on a real Unitree G1, across ψ0, π0.5, and GR00T N1.6.
  • Owes the gain to the calibrated 3DGS frontend — end-task success improves 1.6×–3.25× over a mesh-only baseline trained on the same motions (across backbones), not dataset size.
  • Re-renders one motion dataset for new target scenes, objects, and prompts at over 15× lower cost than re-teleoperating — retaining success under the combined object-and-scene shift, where every default-only baseline collapses to 0–1/10.

Method

A phone scan in. A humanoid skill out. Everything in between is synthesized.

1
Capture Handheld scene video + one photo per object
2
Reconstruct 3DGS background + SAM3D object meshes
3
Simulate & Generate Procedural demos in MuJoCo physics ⊕ calibrated 3DGS render
4
Deploy Fine-tune a VLA → zero-shot on Unitree G1
LEGS pipeline

hover a step — or a part of the pipeline — to highlight it

Two-stage color calibration

A deterministic two-stage calibration aligns the render to the robot's deployment camera — the first stage calibrates the object mesh, and the second is applied to both the mesh and the 3DGS background.

Mesh + 3DGS calibrated render
raw 3DGS + mesh
mesh calibrated
mesh + 3DGS calibrated
Real robot camera image
real camera
Mesh + 3DGS calibrated render
raw 3DGS + mesh
mesh calibrated
mesh + 3DGS calibrated
Real robot camera image
real camera

One episode, many appearances

1 recorded episode motion only — independent of appearance
re-render: ~0.1 GPU-hr per condition
re-render with
 new 3DGS background  new objects  new prompts
wood
blue
white
wood
blue
white
wood
blue
white

Real-Robot Deployment

Policies fine-tuned on LEGS-generated data, deployed zero-shot on a Unitree G1.

Three tasks of increasing difficulty

Task 1 — manipulation (ψ0)

Task 2 — loco-manipulation (ψ0)

Task 3 — long-horizon (GR00T N1.6)

Task 3 across three VLA backbones

GR00T N1.6

ψ0

π0.5

LEGS-AUG: Low-Labor Adaptation to Target Appearances

Motion is recorded independently of appearance, so one recorded dataset re-renders for new target objects and backgrounds. Each new appearance condition (50 episodes) costs ~0.1 GPU-hr versus >1.5 operator-hr for re-teleoperation (over 15× cheaper).

Without re-rendering: the default-only policy collapses

Even on Task 1 — the simplest, stationary pick-and-place — a policy trained only on the default demonstrations (orange→plate on a wooden table) collapses once the objects and prompt change, and loses success even when only the table changes.

failure2×

Scene shift
“place the orange on the plate”

failure2×

Object shift
“put the apple in the box”

failure2×

Scene + object shift
“put the apple in the box”

With LEGS re-rendering: adapted to each target appearance

LEGS-AUG re-renders the recorded motions with the target objects, table, and prompt, so its training set contains the evaluation appearance: this is target-specific adaptation, not generalization to unseen assets.

success

Scene shift
“pick the orange, turn right, and place it on the plate”

success

Object shift
“pick the apple, turn right, and put it in the box”

success

Scene + object shift
“pick the apple, turn right, and put it in the box”

Out-of-distribution object poses

Task 3 with the orange pushed beyond the training distribution.

Far left (out-of-distribution)

OOD probe

Far right (out-of-distribution)

OOD probe

Key Results

Real-robot end-task success across three tasks, three VLA backbones, and four data conditions — 1,230 trials, plus 240 matched-camera trials.

LEGS (200) is best or tied on every task and every backbone

Even at the same data budget, LEGS (50) matches or beats Teleop (50).

Teleop (50) SAM3D (200) LEGS (50) LEGS (200) — ours

ψ0

10
5
0
T1T2T3

GR00T N1.6

10
5
0
T1T2T3

π0.5

10
5
0
T1T2T3

Under the hardest shift (objects + scene), re-rendering wins

(a) Calibrated 3DGS beats mesh-only

SAM3D-aug (200) LEGS-aug (200) — ours
Task 1
60%
100%
Task 2
50%
80%
Task 3
20%
40%

(b) Augmentation beats scale

LEGS (200), default-only LEGS-aug (50)
Task 1
10%
50%
Task 2
10%
40%
Task 3
10%
30%

Q1Can teleoperation-free synthetic data match human teleoperation for VLA fine-tuning?

Yes — on every (backbone, task) cell. LEGS (200) matches or exceeds Teleop (50) across all nine experiments. On the long-horizon Task 3, teleoperation collapses to 0/10 across all three backbones, whereas LEGS achieves up to 6/10.

Q2Is the improvement attributable to dataset size?

No. At a budget-matched 50 demonstrations, LEGS (50) still matches or surpasses Teleop (50) on every experiment, isolating the gain to the data pipeline rather than its scale.

Q3Does the rendering frontend matter, or does mesh-only synthesis suffice?

The calibrated 3DGS frontend improves end-task success by 1.6×–3.25× across the three VLA backbones. With the same motion data, LEGS (200) beats the mesh-only SAM3D (200) baseline on all nine (backbone, task) experiments, and it still leads with the camera in the same automatic mode as the baselines (Task 3: 18/60 vs 6/60).

Q4How efficiently can LEGS adapt to new appearance conditions?

Over 15× cheaper than teleoperation, with task success retained. Each new appearance condition (50 episodes) requires ~0.1 GPU-hr to re-render versus >1.5 operator-hr to re-teleoperate. Under the hardest object-and-scene shift, LEGS-AUG reaches 100 / 80 / 40% on Tasks 1–3, while both teleoperation and unaugmented LEGS fail (0–10%).

BibTeX

@inproceedings{kim2026legs,
  title     = {LEGS: Fine-Tuning Teleop-Free VLAs for Humanoid Loco-manipulation in an Embodied Gaussian Splatting World},
  author    = {Kim, Hojune and Chen, Timothy and Sun, Jiankai and Osterberg, Lars W. and Chen, Qianzhong and Wang, Ke and Schwager, Mac},
  booktitle = {Conference on Robot Learning (CoRL)},
  year      = {2026}
}