If you are comparing two ways to train a robotic arm, start by holding out a task neither system saw during training. ToyBlocks, the simulation environment Ai2 introduced on November 6, 2026, lets researchers make that comparison with blocks on a tabletop. Its value is that you can repeat the test and see which step causes the robot to fail. [1] [2]
What Ai2 makes available
ToyBlocks is the first environment in FoundationWorlds. In the simulation, an arm pushes, flips and stacks blocks. The environment generates task demonstrations, and the repository provides code and tools for measuring success and partial progress. This lets researchers study what happens when they change the training data while keeping a common scenario. Hugging Face hosts an explanation of the project, which was developed by the Allen Institute for AI. [1] [2] [3]
It helps to separate a demonstration from an evaluation. A demonstration shows how to complete an action and can supply training data. An evaluation measures how the system behaves under conditions reserved for testing. If the same setup appears in both, a good result may reflect familiarity with the example and tell you little about performance in a different situation.
A comparison you can repeat
Imagine two systems tasked with placing one block on top of another. I would give both the same training budget and hold out a fixed set of starting positions. One comparison could change how many grasping demonstrations each system sees while keeping the rest of the data the same. Both would then face the same held-out evaluation. That makes it easier to link any performance difference to the change in training.
The share of completed stacks is a useful first measure. To diagnose failures, I would record whether the arm reached the block, grasped it, turned it to the right orientation and released it where intended. Two systems with the same success rate can need different improvements. One might lose the block while moving it, while the other reaches the stack and knocks it over during the final motion.
I would also keep results by starting position, together with the number of attempts and the conditions of each trial. If performance falls when a block starts near the edge of the table, that can guide which examples to add during the next training round. If you use that result to adjust training, keep a separate set of starting positions for the final check. These steps are Actualiza2’s proposal for using the project’s tools.
What can transfer to a physical arm
Simulation makes it easier to repeat conditions and observe failures without reserving a physical arm for every attempt. On a real table, friction, calibration, lighting and the condition of the gripper all matter. A result in the virtual environment can help decide what to test first. To evaluate transfer, I would repeat the same maneuvers on a physical arm in a supervised space and record slips, unexpected contact and interventions as well.
The code and documentation are available in Ai2’s public repository. The sources describe a research and simulation environment. The public repository lets readers inspect the procedure, with no commercial rollout by country to confirm. Availability of a particular arm and physical compatibility are separate questions. Actualiza2 reviewed the project documentation. The comparison outlined here is a proposed test. [1] [3]