You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Released demo dataset (71 trajectories) appears smaller than the dataset described in the paper
#64
Thanks for releasing the code and demos — the DamageSim plugin has been straightforward to get running.
While setting up the IL experiments I found that the Hugging Face release is smaller than what the paper describes, and I wanted to check whether I'm looking in the wrong place.
What's in ut-robin-lab/oopsieverse-demos: 54 HDF5 files, 71 trajectories total. 37 files contain a single demo_0; 17 contain demo_0 and demo_1.
behavior1k/teleop: 26 files, 27 demos (only shelve_item_safe.hdf5 has 2)
robocasa/teleop: 28 files, 44 demos
What the paper describes: Sec. IV states 90 demonstrations across five tasks (Shelve Cereal Box, Lift Egg, Add Firewood, Pour Water, Wipe Countertop), evenly split between the with- and without-health-feedback conditions. The conclusion states 450 demonstrations. Those five tasks are all BEHAVIOR-1K side, where the release has ~11 trajectories combined.
Questions:
Is the IL training set behind Fig. 5 intended to be released separately, or is the HF repo meant to be the complete dataset? The docs describe these files in the context of quickstart_*.py playback, so I may be reading a visualization catalog as a training set.
Sec. IV / conclusion disagree on 90 vs. 450 — is one a typo?
The paper's manipulation is with vs. without live health feedback, but filenames encode _safe / _unsafe, which is an outcome label. Is the collection condition stored in per-demo attributes anywhere? Without it the with/without-feedback split isn't recoverable from the release.
Reproduction (reads HDF5 metadata over HTTP, no full download):
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Thanks for releasing the code and demos — the DamageSim plugin has been straightforward to get running.
While setting up the IL experiments I found that the Hugging Face release is smaller than what the paper describes, and I wanted to check whether I'm looking in the wrong place.
What's in
ut-robin-lab/oopsieverse-demos: 54 HDF5 files, 71 trajectories total. 37 files contain a singledemo_0; 17 containdemo_0anddemo_1.behavior1k/teleop: 26 files, 27 demos (onlyshelve_item_safe.hdf5has 2)robocasa/teleop: 28 files, 44 demosWhat the paper describes: Sec. IV states 90 demonstrations across five tasks (Shelve Cereal Box, Lift Egg, Add Firewood, Pour Water, Wipe Countertop), evenly split between the with- and without-health-feedback conditions. The conclusion states 450 demonstrations. Those five tasks are all BEHAVIOR-1K side, where the release has ~11 trajectories combined.
Questions:
quickstart_*.pyplayback, so I may be reading a visualization catalog as a training set._safe/_unsafe, which is an outcome label. Is the collection condition stored in per-demo attributes anywhere? Without it the with/without-feedback split isn't recoverable from the release.Reproduction (reads HDF5 metadata over HTTP, no full download):
All reactions