HT-GSOM: Dynamic self-organizing map with transience for human activity recognition

Recognition of complex human activities is a prominent area of research in intelligent video surveillance. The current state-of-the-art techniques are largely based on supervised deep learning algorithms. The inability to learn from unlabeled video streams is a key shortcoming in supervised techniques in most current applications where large volumes of unlabeled video data are utilized. Furthermore, the dominant focus on persistence in traditional machine learning algorithms has induced two limitations; the influence of outdated information in memory- guided decision making, and overfitting of acquired knowledge on specific past events, weakening the plasticity of the learning system. To address the above requirements, we propose a new adaptation of the Growing Self Organizing Map (GSOM), formed in a hierarchical two-stream learning pipeline to accommodate unlabeled video data for human activity recognition, which facilitates plasticity by implementing a transience property, without losing the stability of the learning system. The proposed model is evaluated using two benchmark video datasets, confirming its validity and usability for human activity recognition.