Heterogeneous face matching

Abstract

Facial recognition technology faces a complex task in heterogeneous face matching, which involves computing the similarities between face images obtained through different modalities. Overcoming this challenge would enable the efficient matching of a vast collection of frontal photos from various visible face databases, such as passports, driver’s licenses, and mug shots, with face images acquired through alternate modalities, including hand-drawn face sketches, near-infrared photos, three-dimensional faces, and low-resolution photos. Heterogeneous face matching is a vital area of research that holds significant implications for law enforcement and digital entertainment applications, particularly when it comes to facial photo-sketch matching. This thesis puts forward two key contributions to the algorithms used in heterogeneous face matching, with a specific focus on matching face photos to hand-drawn face sketches. The first contribution is a framework designed for matching handdrawn face sketches to face photos. The precision of state-of-the-art face sketch recognition can be significantly improved by utilizing the Image Quality Assessment (IQA-Fusion) technique, which involves the fusion of Multi-Image Quality Assessment metrics. This approach enables the search for criminal suspects’ faces using hand-drawn face sketches based on verbal descriptions of a person’s appearance. The IQA-Fusion technique does not require training, which resolves the issue of the high cost associated with collecting large-scale photo-sketch pairs for training matches based on deep learning. The second key contribution focuses on a technique called cGAN-FSS, which involves using conditional Generative Adversarial Networks to generate facial sketches from facial photographs. The cGAN-FSS framework is capable of producing high-quality face sketch synthesis while maintaining a high level of accuracy in identity recognition. This contribution is considered valuable because it aids witnesses in visually recognizing criminal suspects and helps to bridge the gap between face photos and sketches in the pre-processing phase of facial recognition