Enhancing Facial Emotion Recognition Using Convolutional Neural Networks: Addressing Challenges in Class Imbalance and Generalizability

Abstract

FER is important in today's human-computer interfaces and is used in medicine, education, security and the entertainment world. A hybrid CNN-LSTM model is proposed here to classify facial expressions that show an-ger, disgust, fear, happiness, sadness, surprise or none of them. The dataset was corrected with normalization and improved using rotations, zooming and flipping to help reduce class imbalance. Both convolutional layers and LSTM layers play a part in the architecture of detecting features across time and space. Tests using accuracy, precision, recall and F1-score confirm that the model performs better than many others: 93.25% accuracy, 92.80% precision, 92.50% recall and 92.65% F1-score. These results show that the proposed method is better than CNN (87.10%) and ResNet-50 (89.30%). "Happy" and "Neutral" expressions are recognized with a high level of accuracy, while fewer "Disgust" and "Fear" images are accurately recognized because they are similar and there are not enough examples in the dataset. The good results from the model are not completely reliable since the dataset is too controlled for real-world outcomes. Going forward, it would be useful to add attention mechanisms, make the datasets more inclusive and develop automatic multisensory emotion recognition systems. The work developed a practical framework for FER that can be implemented in real situations.