Face Detection

During my undergraduate degree in Systems Engineering, I worked with a fellow student on a machine learning algorithm to detect faces in images. Given a photo containing a face somewhere in the frame, the algorithm returns the position and bounding box of that face. It’s based on singular value decomposition (SVD), a technique also used in applications like least squares.

Using an approach similar to least squares, we generate a “mean face” by regressing greyscale faces from a dataset of 300 images, each 68×56 px. To do this, we use

where \(\sigma_i\) is the i-th singular value, \(\bf{u_i}\) is the i-th column vector of matrix \(U\), and \(\bf{v_i^T}\) is the i-th row vector of matrix \(V\) from the SVD. By keeping only the first few singular values — making \(k\) much smaller than the full set from our SVD — we reduce the dimensionality of the image matrix. From this reduced representation, we compute the norm between the “mean face” matrix and the reduced matrix of the image we want to classify. Using the test data, we then derive a threshold that determines whether an image is classified as containing a face: in our case \(\varepsilon = 14.0\), since all faces in the test set had \(\varepsilon_{max}\) below \(12.329\), while a reference image without a face scored \(\varepsilon = 22.269\). This lets us automatically classify whether a given image contains a face.

Read the full report in German here

Visualised “mean face”:

Norm of the test dataset of images containing a face, alongside an example of a correctly classified image:

With this approach, we can now locate a face within a larger image using a kernel that scans across the image, comparing each section to the “mean face.” The section with the minimal norm is assigned the highest probability of containing a face.

Skyline image in which the face should be found:

Least-squares deviation for each kernel position scanning the skyline image:

Kernel section identified as most likely to contain a face: