An ensemble approach using a frequency-based and stacking classifiers for effective facial expression recognition

Abstract

Facial Expression Recognition is an essential aspect of human behavior to communicate effectively. A more profound understanding of human behavior, accurate analysis, and interpretation of the emotional content is essential. Hence, facial features play a crucial role as they contain beneficial information about facial expressions. A baseline architecture belonging to the EfficientNet family of models is explored for feature extraction. In this work, two novel strategies, the ensemble model using the frequency-based voting approach (FV-EffNet) and the stacking classifier (SC-EffNet), are proposed to enhance classification results’ performance. The proposed system deals with both profile and frontal pose variations. The combination of deep learning models with a stacking classifier gave the best results of 98.35% and 98.06%, and the frequency-based approach used with the ensemble classifier achieved superior performance of 98.71% and 98.56% on Oulu-CASIA and RaFD datasets, respectively. The experiment results with the proposed methodology showed better performance than previous studies on Oulu-CASIA and RaFD datasets, making it more robust to pose variations. © 2022, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.