Facial Expression Recognition Enhancement Using Convolutional Neural Network

Zahraa Fawzi Hassan; M. N. Al-Turfi; Faiz Al Alawy

doi:10.58564/IJSER.3.2.2024.182

Authors

Zahraa Fawzi Hassan Computer Engineering Department, College of Engineering, Al-Iraqia University, Iraq
M. N. Al-Turfi Electrical Engineering Department, College of Engineering, Al-Iraqia University, Iraq
Faiz Al Alawy Department of Computer Technology Engineering, AL Qalam University College, Kirkuk, Iraq

DOI:

https://doi.org/10.58564/IJSER.3.2.2024.182

Keywords:

Facial Expression Recognition, CNN, KDEF, Data Augmentation

Abstract

Facial expression recognition (FER) technology is quite popular in the fields of computer vision, security monitoring, image classification, and many other related applications. Enhancing the computer's ability to read facial expressions is important for human-computer interaction, as it enables machines to understand and interact with human emotions. In this paper, a modified approach using neural networks (CNNs) is presented to accurately identify unique facial expressions. using the changes to a commonly used 12-layer CNN model to improve its performance in facial expression recognition (FER). The model is trained using dataset images, which allows it to better infer people's emotions from their facial features. To enhance the accuracy of the system, a preprocessing stage is incorporated. This stage involves several operations for data augmentation, including changing the color in images, such as HSV (Hue, Saturation, Value), YCbCr (Luma, Blue-difference Chroma, Red-difference Chroma), etc., to facilitate better interpretation of color information. Additionally, the preprocessing stage refines facial expression recognition by manipulating facial features and adding extra RGB details, improving the visual information provided by the images. The study specifically focuses on evaluating the effectiveness of this approach using the KDEF database. The KDEF database contains standardized images of facial expressions, making it suitable for assessing the performance of the proposed system. By combining the modified CNN model, training with images, and the preprocessing operations, the system's performance was significantly enhanced. As a result, it achieved recognition rates of up to 95%, indicating a notable improvement in accurately identifying unique facial expressions compared to previous approaches.

References

Dalvi, C., Rathod, M., Patil, S., Gite, S., & Kotecha, K. , "A survey of ai-based facial emotion recognition: Features, ml & dl techniques, age-wise datasets and future directions," Ieee Access, vol. 9, pp. 165806-165840, 2021.

Mehrabian, A., & Ferris, S. R., "Inference of attitudes from nonverbal communication in two channels," Journal of consulting psychology, vol. 31, no. 3, p. 248, 1967.

Mohammadpour, M., Khaliliardali, H., Hashemi, S. M. R., & AlyanNezhadi, M. M., " Facial emotion recognition using deep convolutional networks.," In 2017 IEEE 4th international conference on knowledge-based engineering and innovation (KBEI),IEEE., pp. 0017-0021, (2017, December).

Khan, A. R., " Facial emotion recognition using conventional machine learning and deep learning methods: current achievements, analysis and remaining challenges," Information, vol. 13, no. 6, p. 268, 2022.

Moret-Tatay, C., Wester, A. G., & Gamermann, D., " To Google or not: Differences on how online searches predict names and faces," Mathematics, vol. 8, no. 11, p. 1964, 2020.

Ozdemir, M. A., Elagoz, B., Alaybeyoglu, A., Sadighzadeh, R., & Akan, A., "Real time emotion recognition from facial expressions using CNN architecture.," In 2019 medical technologies congress (tiptekno),IEEE, pp. 1-4, 2019.

H. Mohammed, M. Nasser Hussain, and F. Al Alawy, "Feature Extraction Techniques for Facial Expression Recognition (FER)," IJSER, vol. 2, no. 3, p. 32–40, Sep. 2023..

Elarbi-Boudihir, M., Rehman, A., & Saba, T. , "Video motion perception using optimized Gabor filter.," International journal of physical sciences, vol. 6, no. 12, pp. 2799-2806, 2011.

H. Mohammed, M. Nasser Hussain, and F. Al Alawy,, "Facial Expression Recognition: Machine Learning Algorithms and Feature Extraction Techniques," IJSER, vol. 2, no. 2, p. 23–28, Jun. 2023..

Liang, D., Liang, H., Yu, Z., & Zhang, Y. , "Deep convolutional BiLSTM fusion network for facial expression recognition," The Visual Computer, vol. 36, pp. 499-508., 2020.

Ko, B. C. , " A brief review of facial emotion recognition based on visual information," sensors, vol. 18, no. 2, p. 401, 2018.

Recht, B., Roelofs, R., Schmidt, L., & Shankar, V., "Do imagenet classifiers generalize to imagenet?," In International conference on machine learning ,PMLR., pp. 5389-5400, (2019, May). .

Ahmed, T. U., Hossain, S., Hossain, M. S., ul Islam, R., & Andersson, K. , "Facial expression recognition using convolutional neural network with data augmentation," In 2019 Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) and 2019 3rd International Conference on Imaging, Vision & Pattern Recognition (icIVPR), IEEE., pp. 336-341, 2019.

Sajjad, M., Zahir, S., Ullah, A., Akhtar, Z., & Muhammad, K., "Human behavior understanding in big multimedia data using CNN based facial expression recognition," Mobile networks and applications, vol. 25, pp. 1611-1621., 2020.

Eng, S. K., Ali, H., Cheah, A. Y., & Chong, Y. F., " Facial expression recognition in JAFFE and KDEF Datasets using histogram of oriented gradients and support vector machine.," In IOP Conference series: materials science and engineering, IOP Publishing., vol. 705, no. 1, p. 012031, 2019.

Hussain, S. A., & Al Balushi, A. S. A. , "A real time face emotion classification and recognition using deep learning model.," In Journal of physics: Conference series , IOP Publishing., vol. 1432, no. 1, p. 012087, 2020.

Zhang, L., Xu, C., & Li, S., "Zhang, L., Xu, C., & Li, S. Facial expression recognition of infants based on multi-stream CNN fusion network.," In 2020 IEEE 5th International Conference on Signal and Image Processing (ICSIP)IEEE., pp. 37-41, 2020.

Liu, Yuanyuan, Wei Dai, Fang Fang, Yongquan Chen, Rui Huang, Run Wang, and Bo Wan. , "Dynamic multi-channel metric network for joint pose-aware and identity-invariant facial expression recognition," Information Sciences, vol. 578 , no. 2021, pp. 195-213, 2021.

Wafi, M., Bachtiar, F. A., & Utaminingrum, F. , "Feature extraction comparison for facial expression recognition using adaptive extreme learning machine," International Journal of Electrical and Computer Engineering, vol. 13, no. 1, pp. 1113-1122, 2023.

KDEF database, "kaggle," [Online]. Available: https://www.kaggle.com/datasets/tom99763/testtt.

Huang, Y., Chen, F., Lv, S., & Wang, X., " Facial expression recognition: A survey," Symmetry, vol. 11, no. 10, p. 1189, 2019.

Shorten, C., & Khoshgoftaar, T. M. , " A survey on image data augmentation for deep learning," Journal of big data, vol. 6, no. 1, pp. 1-48., 2019.