Elite Chess and the Data Void: When the Analyst's Table Has Nothing to Count

In a broadcast a few years ago, I was assigned to commentate a game for which the organisers sent me exactly two things: the names of the two players and the result. No PGN file, no time spent per move, not a single line of engine analysis. I opened my own spreadsheet; the ACPL column sat empty. The host asked me which side had prepared the opening better. I said I needed thirty minutes to rebuild the game from the first move. A fellow guest answered immediately, fluently, decisively, without pausing for breath. Nobody in that studio had enough data to verify either answer.

That is the permanent condition of elite chess analysis: a great deal of language produced on a very thin foundation of data. What worries me more is that most viewers have no way to tell which of the two men was speaking from evidence and which from reputation.

The vocabulary of the modern chess world has become among the most technical in sport. Audiences hear terms like average centipawn loss (ACPL), engine match rate, analysis depth, number of theoretical moves beyond the book, and clock time remaining when the endgame begins. Match reports in Vietnam and abroad deploy them as if they were default background data — available, reliable, settled.

The operational reality is different. A standard tournament publishes results and move scores but rarely the engine depth used for evaluation, the software version, or when the analysis was run. An ACPL computed with Stockfish at depth 18 differs from an ACPL computed by a light engine running on a laptop in the hall. The same game and the same player can produce two numbers far apart enough to reverse the conclusion.

I have spent much of my career digitising handwritten notebooks. When I moved 2,400 games from paper into a spreadsheet, I found something simple: observations recorded with numbers survived time; observations recorded only with adjectives did not. Everything on a chessboard is data waiting for a reader — if you are willing to sit down.

Three layers of absence appear in nearly every chess analysis I read or hear each week.

The first is the metric layer. ACPL is an average, and every average conceals a distribution. A game with an ACPL of 12 might be one in which the player played with extreme safety, traded pieces repeatedly and steered into a drawn endgame; it might equally be one in which he found a single key move and erred through everything else. An average cannot distinguish the two. A writer working only from ACPL will assign a character to a game that the game does not have. An average answers the question of whether the play was clean, not how the win was built — and those two questions require two different datasets.

The second is the opening layer. Calling a player well prepared usually rests on a move that has not appeared in a database, or has appeared in very few games. Public databases lag. An idea treated as new at move 14 may have been played at move 10 in an open tournament three weeks earlier, in too few games to pass the analyst's filter. I require any opening analysis to state its sample size before drawing a conclusion. A new idea across three games is an experiment, not yet a weapon.

The third is the format layer. Major events increasingly rely on rapid and blitz tiebreaks to settle a title when the classical series ends level. As tournament policy, this is a reasonable way to produce a champion on the day. As data, it is a large displacement: ACPL in blitz runs far higher than in classical chess, not because players understand less but because they have less time to think. When a classical title is decided by such games, the statistical meaning of everything preceding it is partly undone. Match reports cover the player lifting the trophy and rarely state plainly that the decisive portion of the title was contested in another format altogether.

I once rebuilt four games of a national team using old software and a stopwatch, after a remark of mine was quoted out of context and caused an uproar. The result showed the team rotating three structures within a single match. There was nothing mysterious about it. It only required counting, and sitting long enough to finish counting.

The counterintuitive angle here: a data void is itself a finding worth publishing, rather than a shortcoming to paper over. When a model returns insufficient information across every dimension — technical, player profile, format, risk and public narrative — the correct conclusion is not that analysis was impossible, but that the source being circulated contains no verifiable content. That is a valuable conclusion, and it deserves to be printed.

What concerns me more is the reflex in the opposite direction: stuffing the table to fill it. From a single game a writer can cite ACPL, theoretical moves, Elo ratings, head-to-head history, White's win rate and time-pressure indices. Every one of them is real. If the collection answers no specific question, it is decoration. Engine analysis does not create mistakes; it strips the mask from those who believed they were making none. I no longer believe in miracles on the board; I believe in the conversion rate of an advantage — and that rate has to be measured on a large enough sample, against a definition of advantage written down before the results are seen.

One principle I have kept for years: after every major tournament cycle, I take the old model out and re-examine it, not to see whether it was right or wrong, but to see whether it still measures what I need measured. Tools change, the principle holds. But the principle must also face inspection.

Chess analysis improves when practitioners publish what they do not know: sample size, engine version, depth, and the categories that must be marked insufficient information. Credibility comes from stating clearly the boundary of what has been measured, not from saying much. If an analysis of the year's biggest game cannot produce a single verifiable fact, what exactly is the reader reading?