WORLDTECHAi2 Releases ToyBlocks, a Simulator for Studying How Robots Learn and Fail
Ai2 has put out ToyBlocks, a simulated tabletop setting built to show how robots learn and where they fall short. According to one outlet's report, the release brings together an open robot simulation, generated demonstrations, two trained models and evaluation tools.
What the environment contains
The setting places a simulated arm and labeled blocks on a tabletop. Alongside pushing, flipping and stacking come search, sorting and arrangement tasks. In some of these, the letter or picture a robot needs may be hidden, so the machine must turn a block over before it can determine where that block belongs.
A task-and-motion planner chooses skills and sequences them to produce labeled demonstrations. ToyBlocks is the first environment in Ai2's FoundationWorlds framework, which is meant to make robot-learning experiments repeatable and failures easier to trace.
Automated demonstrations and open models
Skills are created through reinforcement learning and motion planning in Nvidia's Isaac Lab. Generation can draw on exact block positions and orientations taken from the simulator instead of camera estimates, automating behaviors that would otherwise require a person to teleoperate a physical robot. Ai2 says hundreds of plans can run at once on one GPU, at up to 24 times real-time performance. Researchers can weight the resulting training mixtures toward particular skills and test policies on tasks held back from training.
The initial release includes 85,336 long-horizon demonstrations, each made up of roughly 6 to 12 skill demonstrations depending on the set. Ai2 reports that they cover nearly 1,600 hours of robot execution and 144 million frames. The public dataset is offered as LeRobot data in Parquet format under an ODC-BY license, divided into easy, medium and hard subsets. The main code carries an Apache-2.0 license, while specified submodules and downloaded assets fall under their own licenses.
Ai2's model cards split the released policy into two levels. Molmo2-4B-tb, a model of roughly 4.5 billion parameters, takes a front-camera image and a text goal and outputs a short plan, a next action and a target point. ManiFlow-tb, with 322.5 million parameters, takes front and gripper images, robot state, task information and a target heatmap and generates chunks of actions.
Generalization results
The spatial experiments cover Stack, Unstack, Flip and Rotate and distinguish interpolation within the training range from extrapolation beyond it. Examples include training with blocks in one region of the workspace and testing in another, or withholding particular stack heights.
Ai2 reports weak initial generalization for ManiFlow, π0.5 and MolmoAct 2 despite large-scale pretraining, and concludes that generalization in action models remains far from solved. The agentic tests point to a different weakness: the 30 scene configurations contained 4 to 10 blocks that had appeared during training.
Open questions
- How the agentic tests perform on blocks beyond the 4 to 10 seen during training remains unclear from the reported facts?
- Whether the reported generalization gaps also appear on physical robots is not established by the facts?