High Schooler Creates Minecraft AI Benchmark to Test Generative Models
A 12th-grader built MC-Bench, a site where AI models compete in Minecraft build-offs to test their capabilities.
Matilda
High Schooler Creates Minecraft AI Benchmark to Test Generative Models
As conventional AI benchmarking techniques prove inadequate, AI builders are turning to more creative ways to assess the capabilities of generative AI models. For one group of developers, that’s Minecraft, the Microsoft-owned sandbox-building game. Image:Minecraft The website Minecraft Benchmark (or MC-Bench) was developed collaboratively to pit AI models against each other in head-to-head challenges to respond to prompts with Minecraft creations. Users can vote on which model did a better job, and only after voting can they see which AI made each Minecraft build. For Adi Singh, the 12th-grader who started MC-Bench, the value of Minecraft isn’t so much the game itself, but the familiarity that people have with it — after all, it is the best-selling video game of all time. Even for people who haven’t played the game, it’s still possible to evaluate which blocky representation of a pineapple is better realized. “Minecraft allows people to see the progress [of AI development] much more easil…
