A Go program amateurs can beat found KataGo's blind spot — and beat the superhuman AI

KataGo, the open-source Go AI that beats top human players, learned the game from millions of games against itself. In November 2022, a team of AI researchers including Adam Gleave, then a PhD candidate at UC Berkeley, published a method to defeat it: an adversarial policy, built from a neural network plus Monte-Carlo Tree Search, that exploits blind spots left by that training.

During a match, Gleave explains, the adversary first claims a small corner of the board and lets KataGo take the rest, seeding its territory with easy-to-capture stones. 'This tricks KataGo into thinking it's already won,' he says — but the bottom-left territory doesn't actually contribute to its score, because the presence of black stones means it isn't fully secured.

Convinced it is winning, KataGo plays a pass; two consecutive passes end the game, and the tally counts only the adversary's corner territory. The trick works despite the adversary being so weak that human amateurs can beat it relatively easily — its sole purpose is to attack an unanticipated vulnerability.

Gleave says the result shows AI systems that seem to perform at human level 'are often doing so in a very alien way, and so can fail in ways that are surprising to humans' — entertaining in Go, but a warning for safety-critical systems, underscoring the need for automated testing that finds worst-case failure modes rather than average-case performance.

Don't outplay the machine — mislead it: hold one corner, let the AI take 'winning' but unsecured territory, and it passes early, ending the game on the bluff.

It defeats a superhuman game-playing AI not by playing better Go but by exploiting how the AI scores the game.

The winning mechanism is a bluff: convince the opponent it has already won, then end the game while its territory is still unsecured.

The adversary is bad at Go by design — amateurs beat it — isolating the result as a vulnerability find, not a strength gap.

It generalises a warning for every deep-learning system: off-distribution inputs can flip behaviour in ways average-case testing never sees.

Superhuman average performance can hide alien reasoning: a system that passes every benchmark may fail on off-distribution inputs, so test for worst-case behaviours, not average-case scores.

The paper's authors framed the result as an argument for better automated testing of AI systems against worst-case failure modes, with applications beyond games — a self-driving car, for example, could likewise be tricked by scenarios outside its training set. Gleave noted that KataGo's exposed strategy is 'likely' one of many such blind spots.