Online video scene clustering by competitive incremental NMF

Efficient clustering and categorizing of video are becoming more and more vital in various applications including video summarization, content-based representation and so on. The large volume of video data is the biggest challenge that this task presents, for most the clustering techniques suffer from high dimensional data in terms of both accuracy and efficiency. In addition to this, most video applications require online processing; therefore, clustering should also be done online for such tasks. This paper presents an online video scene clustering/segmentation method that is based on incremental nonnegative matrix factorization (INMF), which has been shown to be a powerful content representation tool for high dimensional data. The proposed algorithm (Comp-INMF) enables online representation of video content and increases efficiency significantly by integrating a competitive learning scheme into INMF. It brings a systematic solution to the issue of rank selection in nonnegative matrix factorization, which is equivalent to specifying the number of clusters. The clustering performance is evaluated by tests on TRECVID video sequences, and a performance comparison to baseline methods including Adaptive Resonance Theory (ART) is provided in order to demonstrate the efficiency and efficacy of the proposed video clustering scheme. Clustering performance reported in terms of recall, precision and F1 measures shows that the labeling accuracy of the algorithm is notable, especially at edit effect regions that constitute a challenging point in video analysis.