SimVLA: Zero-shot Sim-to-Real VLA Learning for Mobile Manipulation

Project page · Paper · Video · Datasets

SimVLA overview

Generate procedural kitchens, compose mobile-manipulation tasks from reusable skills, and build robot-action and grounded visual-question-answering datasets. This is the research code behind SimVLA's zero-shot sim-to-real mobile-manipulation work.

Installation

Full step-by-step instructions live in docs/installation.md. The hash-locked kitchen, Anubis, RB-Y1, and AI Worker assets are available as a separate SimVLA-assets download. For an existing research checkout, the verified staging recipe validates and links the source trees without duplicating the large asset corpus.

python -m venv .venv
source .venv/bin/activate
python -m pip install -e '.[scenes]'
simvla demo --output-dir outputs/quickstart

Documentation

1. Kitchen Scene Generation

Generate one of five procedural layouts on a CPU:

simvla scene --layout l_shaped --seed 1 --output outputs/kitchen.glb

For USD generation with Isaac Sim, see Kitchen Scene Generation.

2. Goal Generation

Build task JSON from registered skill APIs, validate it, and bind it to a kitchen:

simvla skills
simvla init-task --from bowl_to_drawer --name my_task --output outputs/my_task.json
simvla validate outputs/my_task.json

See task authoring and Goal Generation, including instructions for adding skills and using AI Worker.

3. Hybrid System Identification

Fit robot stiffness and damping against a real recording.

See Hybrid System Identification.

4. SimAction Generation

Collect synchronized robot actions and multi-view observations in IsaacLab.

See SimAction Generation.

5. SimVQA Generation

Convert captured RGB and segmentation observations into grounded questions and answers:

simvla vqa examples/vqa --output outputs/questions.jsonl

See SimVQA Generation.

6. SimVLA Training

The paper uses SimAction and SimVQA pre-training followed by SimAction and SimDeploy post-training. The training recipe and checkpoint are not included in this repository. See the reproducibility map.

7. SimDeploy Generation

SimDeploy generates deployment-oriented simulator data for post-training. Its public training adapter and checkpoint are not included. See Training and SimDeploy.

8. Evaluate in IsaacLab

Run a physics and camera smoke test before evaluation:

KITCHEN_NUMBER=99001  # choose a kitchen excluded from training
SUB_NUMBER=00         # one of its generated variants
simvla run smoke --repo-root . -- \
  --task "Isaac-Kitchen-v${KITCHEN_NUMBER}-${SUB_NUMBER}" --seed 0 --headless

Task IDs follow Isaac-Kitchen-v<kitchen_number>-<sub_number>. For zero-shot evaluation, choose a kitchen number excluded from training. See Evaluate in IsaacLab.

Citation

If you use SimVLA in your research, please cite:

@misc{baik2026simvla,
  title={SimVLA: Zero-Shot Sim-to-Real VLA Learning for Mobile Manipulation},
  author={Kyoungin Baik and Youngwoon Lee},
  year={2026},
  eprint={2610.11248},
  archivePrefix={arXiv},
  primaryClass={cs.RO},
  url={https://arxiv.org/abs/2610.11248},
}

License

See LICENSE.

Contributing

See CONTRIBUTING.md.