A video-based door monitoring system using local appearance-based face models

In this paper, we present a real-time video-based face recognition system. The developed system identifies subjects while they are entering a room. This application scenario poses many challenges. Continuous, uncontrolled variations of facial appearance due to illumination, pose, expression, and occlusion of non-cooperative subjects need to be handled to allow for successful recognition. In order to achieve this, the system first detects and tracks the eyes for proper registration. The registered faces are then individually classified by a local appearance-based face recognition algorithm. The obtained confidence scores from each classification are progressively combined to provide the identity estimate of the entire sequence. We introduce three different measures to weight the contribution of each individual frame to the overall classification decision. They are distance-to-model (DTM), distance-to-second-closest (DT2ND), and their combination. We have conducted closed-set and open-set identification experiments on a database of 41 subjects. The experimental results show that the proposed system is able to reach high correct recognition rates. Besides, it is able to perform facial feature and face detection, tracking, and recognition in real-time.