Ai2’s open ToyBlocks is a simulated arm-and-blocks world with 85,336 demos. Its robots spell words about half the time. The scores are self-reported.
Tag: Ai2
Ai2’s open StateBench tests how models track changing values in long text. Frontier models fall to 13% or lower on the hardest level.