Advanced Chessboard Recognition and Optical Notation Yielding Mechanism (Acronym)

During my exchange semester in Norway at the University of Agder (UiA), I took the course Machine Vision. The course focused on the concepts of image processing and working with tools like OpenCV to solve problems with Python.

For our final projects, I and two fellow students developed the Advanced Chessboard Recognition and Optical Notation Yielding Mechanism (or ACRONYM for short), which aims to convert an image of an over-the-board chess game into a digital representation of the current position. The user should be able to take a picture with their mobile phone, which is then converted into the Forsyth-Edwards chess notation system (FEN).

First we use different filters, colour spaces and edge detection to detect the corners of the chess board. Then we use edge and line detection and a self-written algorithm to find the corners of each square. Using the square corner points, we cut the square to size and pre-filtered the empty squares using the Shannon-entropy of the grayscale spectrum of the square. The non-empty square then where feed into a pre-trained CNN using VGG16. The CNN returns a prediction for colour and piece for each non-empty square. With the prediction from the CNN and the knowledge about the empty squares we are able to define the position of all the pieces on the board and convert them to FEN notation. The predictions had an accuracy of about 90%. In order to achieve such high accuracy, we had to fixate the angel from which the picture has to be taken.

Here an example of the app with input image and returned FEN notation:

Read the full report here