A new technique based on 3D convolutional neural networks and filtering optical flow maps for action classification in infrared video

Abstract

Human action in video sequences provides three-dimensional spatio-temporal signals that characterize both visual appearance and motion dynamics. The aim of this work is to recognize human action in infrared video by focusing mainly on dynamic information. We developed a new technique based on deep 3D convolutional neural networks (3D CNNs) that take optical flow maps as input. Our approach consists mainly of three parts: 1) computation of optical flow maps; 2) filtering of these maps, using an entropy measurement in order to increase the classification rate and reduce the run time by eliminating sequences that do not contain human action; and 3) classification using 3D CNN. The experimental results obtained by our approach on the InfAR dataset show considerable improvement in comparison with results obtained by existing models.