My Chess Improvement Process: Step 1 - Data Mining

This post is part of a series describing the process I am using to attempt improvement in chess. The main post gives an overview of the process while this article explains one of the steps in detail.

The first step on our chess improvement journey is to understand factually the strengths and weaknesses in our game. I am speaking from lifelong experience when I say that this is difficult to intuit. It is simple to remember recent blunders and draw conclusions that are not aligned with your performance over a longer period of time. The cognitive term for this is Recency Bias.

So, the question we are faced with is: How do you measure your chess ability?

There are a great many approaches for this, and my choice is not necessarily the best fit for everyone.

My approach is to identify all mistakes from my competitive chess games with a classical time control. Basically, taking the idea of the "List of Mistakes" that Axel Smith describes in the excellent book Pump Up Your Rating, and turning it up a few notches further by adding quantifiable metadata for each mistake. 

I selected games I have played in rated OTB games (tournaments and league games) as well as online games which were part of a tournament. The important thing about this selection process of games is that it should be games played with serious focus and deep concentration, so that the quality of the data can be relied upon. If the data quality that goes into the analysis is low, the analysis of the data will be useless to you. Garbage in, garbage out.

As we now have a list of games to extract mistakes from, we need to define what constitutes a mistake. My approach is to import each game on lichess, have the server analyze it and present to us a list of inaccuracies, mistakes and blunders.

For each of the identified sub-optimal moves, I manually add metadata, such as:

• the severity of the error,
• the type of mistake (tactical, positional)
• the sub-type of mistake (for example candidate move, shallow calculation, exchange, etc.)
• the phase of the game (opening, middlegame, endgame)
• comment

This exercise in itself is very useful, in that it forces you to analyze your errors systematically. I think there are lessons to be learned just by noting down the errors in a list. This is the reason that I scrapped attempts at automating this process using code. By actually making this a labor intensive process, I am picking up insights just by getting the source data for our analysis in place.

The game itself will also be added to a separate list, with information such as color, result, tournament, date, opponent Elo, and also the evaluation of the position at the end of the opening phase and middlegame phase, as defined by the lichess analysis.

One important detail to consider is in which unit to quantify the error, as well as the evaluation of a position. In the old days we used ECO symbols, today we are used to engine evaluations, but both of these approaches are not optimal when it comes to quantifying an error. For example, let's say you are playing a move, which reduces the engine evaluation by 1.0. There is a huge difference between the loss of winning chances if the initial evaluation is 0.5 or 4.5. In the former case (the evalution goes from 0.5 to -0.5) the expected score is reduced by 14%, while in the latter case (the evaluation goes from 4.5 to 3.5) the loss is only 5%. This conversion between engine evaluation and winning chances assumes that the players are of equal strength, and the formula could be improved for uneven matchups.

For this reason, I am quantifying each mistake in the loss of my expected score, so a 14% error means that on average I am expected to score 0.14 points less after playing this erroneous move.

In the next post I will dive into the exciting world of exploring this data to find insights and actionable information!