Categories
AI News

Ai2’s ToyBlocks Teaches Robots on a Tabletop. Spelling Is Still Hard

Ai2’s open ToyBlocks is a simulated arm-and-blocks world with 85,336 demos. Its robots spell words about half the time. The scores are self-reported.

Categories
AI News

x = y Is Where Even Frontier Models Lose Track, Says Ai2’s StateBench

Ai2’s open StateBench tests how models track changing values in long text. Frontier models fall to 13% or lower on the hardest level.