NEEDLEWORK: Offline Rewriting of Robot Data with Verified Local Stitches

What happened

arXiv:2610.02339v1 Announce Type: new Abstract: Robot demonstrations may contain useful behavior even when individual episodes are inefficient or unsuccessful. First, NEEDLE identifies and creates connections that bypass suboptimal detours, broaden action coverage, and augment the original dataset with failed trajectories, using only RGB images, proprioception, and episode-level outcomes, without new environment interaction or privileged object state.

Trajectory stitching offers a way to compose these behaviors into improved training data, but identifying useful connections and verifying their feasibility is difficult in high-dimensional robot data, where many prior methods rely on low-dimensional state representations. On real-robot tasks, NEEDLE improves success rate over the strongest baseline on each task by an average of 21 percentage points. Videos and supplementary materials are on https://needle-work.github.io/.