Multi-Biometric Person Identification System Using Face and Gait Fusion: A Deep Learning Approach

Abstract

Automatic authentication of people has always been a challenging task especially when it has to deal with the large datasets and the robustness against the factors affecting recognition such as pose variation, subject to camera angle, illumination, poor quality data and occlusion etc. Neural networks in particular have become very popular nowadays because of two things i.e. faster computers and large datasets. So, deep learning and neural networks prove to be a great remedy for above problems. Hence, we have designed an algorithm to identify people at a distance by fusing their gait and face biometrics using a 13-Layer Deep Convolutional Neural Network (DCNN). We utilized the concept of Gait Energy Images (GEIs) to represent the characteristics of human gait. The GEIs and face sequences from same individual are firstly resized into vectors after some sort of pre-processing. Then, both the vectors are fused and the output is fed to the DCNN for feature extraction and classification. Our proposed DCNN is composed of three triplets of Convolution, ReLU and Max-Pooling Layers followed by a Fully Connected Layer, a SoftMax Regression Layer and a Classification Layer. The proposed work is tested upon three publicly available databases i.e. CASIA Gait B, ORL Face, and FEI Face Datasets. A maximum accuracy of 98.75 % is achieved when ORL Face Database is fused with CASIA Gait B Database and 97.50 % accuracy is achieved when FEI Face Dataset is fused with CASIA Gait B Database.We have also tested our model with three noise attacks to both face and gait test images i.e. Salt and Pepper, Gaussian and Speckle Noises. We utilized the median filter to de-noise the images affected with Salt and Pepper noise and mean filter to de-noise the images affected with Gaussian Noise and Speckle Noise both. A recognition accuracy of 97.5 %, 93.75 % and 95 % for the first experiment and 97.18 %, 95.97 % and 95.56 % for the second experiment is achieved in presence of Salt and Pepper, Gaussian and Speckle Noise respectively. We were also able to reduce the computational time as our model took only 3.5minutes to train the network for 1st experiment and around 11 minutes for 2nd experiment. To further increase the recognition accuracy in future, we will try to combine our proposed model with some of the existing popular feature fusion algorithms such as Canonical Correlation Analysis (CCA) and Discriminant Correlation Analysis (DCA).