VibeusPokémon Red makes the starter choice matter before the first gym.... - Vibeus
Pokémon Red makes the starter choice matter before the first gym. PokéBench gives every model a fresh save and the same prompt, then shows one screenshot per turn with the game state and on-screen text; there are no map hints or pathfinding help. Its public leaderboard reads milestones from the game’s RAM.
Opus 5.5 walked past the first ball to pick Squirtle for the Brock matchup. It beat him on turn 271 in 1h 6m for $8.44, without a blackout. Fable 5.1 took the closest ball—Charmander, a weaker matchup into Brock’s Rock types—but entered the gym at turn 230, two turns ahead of Opus. Then the Jr. Trainer’s Sandshrew wiped its team on turn 246. The same Sandshrew did it again at 362. At turn 363, Fable recognized brute force was not working and had $169 in-game money left. It planned to catch a Mankey on Route 22, never went, and instead trained in Viridian Forest; Charmeleon arrived at turn 511, and Brock fell at 554. That run cost $99.76 and had three blackouts. The Reddit post says 3h 17m; the linked run card shows 3h 16m.
The board currently lists 19 runs: GPT-6 Astra first (246 turns, $13.49), Opus second (271, $8.44), Gemini 3.8 Flash third (279, $3.70), and Fable seventh (554, $99.76). PokéBench says scores come from game memory, with the rules, harness, and turn-by-turn logs public.
The author later clarified this is a fun personal test, not a lab benchmark. The prompt shown to the models only set the goal of beating the game; it did not mention the 1,000-turn budget or that the leaderboard run ends at Brock. It is a vivid pair of runs, not a general model ranking.