Robotics SINGLE SOURCE

inr2action stores robot episodes as fixed-size weight deltas for recall and action

First reported 22 Oct, 04:00 UTCSource: arXiv Robotics (cs.RO)Last editorial activity 22 Oct, 04:00 UTCWrite a comment ↓

An industrial machine in a glass factory showcasing automated packaging in Dar es Salaam. — Illustrative photo.Photo by Keegan Checks on Pexels

A preprint on arXiv proposes keeping long robot episodes as a single set of neural weight changes of fixed size. According to the paper, a language model can answer questions about past episodes and a decoder can predict commands, without searching stored video frames when a query arrives.

How the memory is built

The method fits a neural implicit representation to the video of an episode, starting from a shared initialization obtained through meta-learning. The difference between the fitted weights and that starting point, the weight delta, is what the authors treat as the memory.

According to the abstract, the size of this delta does not change with episode length. An episode of a hundred frames and one of a hundred thousand frames yield deltas of identical size, which the authors present as memory tied to essential information rather than to execution time.

Language and action outputs

A learned projector converts the weight delta into token embeddings, the numerical inputs a language model reads. These embeddings condition two components: a language model that answers questions and a decoder that regresses commands jointly. Neither component retrieves frames when a query comes in.

On the RoboMME benchmark, the language model is reported to exceed chance on every question type, and the decoder reaches a mean R² of 0.51.

Control runs point to the initialization

The authors describe two control experiments. Replacing the delta with matched noise lowers performance by 11.4 points. Weights that start from a random origin instead of the meta-learned initialization cannot be decoded, which the paper cites as evidence that the shared starting point is essential for encoding episode content.

The preprint is marked as a new submission and has so far been reported by one outlet, arXiv Robotics (cs.RO), which is the only source for these findings.

Open questions

How well does the approach perform on robotic tasks beyond the RoboMME benchmark?
How is the shared meta-learned initialization obtained, and how much does it depend on the training data?

Key facts

The arXiv preprint 2610.28882v1 describes a memory approach for long-horizon robot operation that scales with essential information rather than execution time.

A neural implicit representation is fitted to episode video, starting from a shared, meta-learned initialization, and the resulting weight delta serves as the memory.

The weight delta keeps a fixed size whether an episode spans a hundred or a hundred thousand frames.

A learned projector maps the delta into token embeddings that condition a language model for question answering and a decoder for joint command regression, with no frame retrieval at query time.

On RoboMME, the language model exceeds chance across all question types, and the decoder reaches a mean R² of 0.51.