Klasifikasi Emosi Pada Raut Wajah Pelajar Menggunakan Ekstraktor Fitur Face Mesh Dan Metode Support Vector Machine
DOI:
https://doi.org/10.25126/jtiik.124Kata Kunci:
emotion recognition, facial landmark, hyperparameter tunning, SVM, face meshAbstrak
Dalam lingkup pembelajaran, rasa emosional dan perhatian memegang peranan penting dalam keterlibatan pelajar terhadap proses pembelajaran yang sedang berlangsung. Emosi pelajar menimbulkan reaksi afektif terhadap proses pembelajaran, seperti boredom, engagement, confusion, dan frustration. Reaksi afektif tersebut dapat digunakan sebagai tolok ukur dalam melakukan evaluasi kegiatan pembelajaran. Pengenalan emosi dapat dilakukan dengan pengamatan citra wajah, namun pemrosesan sebuah citra memerlukan sebuah model yang dapat melakukan klasifikasi emosi berdasarkan raut wajah secara tepat. Berdasarkan permasalahan tersebut, penelitian ini bertujuan untuk membangun sebuah sistem pengenalan emosi melalui raut wajah pelajar dengan ekstraktor fitur Mediapipe Face Mesh dan Support Vector Machine (SVM). Proses ekstraksi frame dan ekstraksi fitur dilakukan untuk mendapatkan total 1404 titik tiga dimensi facial landmark untuk kemudian diklasifikasikan dengan menggunakan SVM. Untuk meningkatkan kinerja klasifikasinya, dilakukan optimasi algoritma SVM melalui hyperparameter tuning dan Grid Search Cross Validation untuk menghasilkan kombinasi parameter model dengan kinerja terbaik. Hasil yang diperoleh adalah 53%, mengalami peningkatan 25% dibandingkan dengan model standar tanpa proses hyperparameter tuning yang menunjukkan bahwa hyperparamter tuning memiliki pengaruh terhadap kinerja model. Selain itu, terdapat titik-titik facial landmark yang berperan dalam klasifikasi emosi berdasarkan hasil analisis, yaitu titik yang berada di sekitar mata.
Abstract
In the context of learning, emotion and attention are significant factors influencing the learner's engagement with the ongoing learning process. The affective reactions of learners to the learning process, which may include boredom, engagement, confusion, or frustration, can be influenced by their emotional state. Such affective reactions may be employed as benchmarks for the evaluation of learning activities. The emotions can be rcognized by analyzing the image of human face. However, image processing needs a model that can accurately categorize emotions based on facial expressions. This research aims to address these issues through the construction of an emotion recognition system based on student facial expressions using the Mediapipe Face Mesh feature extractor and Support Vector Machine. First, a frame extraction and feature extraction process was conducted to obtain a total of 1,404 three-dimensional facial landmark points as input data. Subsequently, the SVM algorithm was optimized through hyperparameter tuning and Grid Search Cross Validation to produce a combination of model parameters with the best performance. The resulting value was 53%, representing a 25% increase compared to the standard model without hyperparameter tuning, which demonstrates that hyperparameter tuning has a significant impact on model performance. Additionally, the analysis revealed that certain facial landmark points, particularly those around the eyes, play a crucial role in emotion classification.
Downloads
Referensi
BHATIA, S., TOMAR, U. AND JAIN, A.V., 2021. Comparing SVM and Neural Networks’ performance in Face Detection. In: 2021 International Conference on Intelligent Technologies (CONIT). [online] 2021 International Conference on Intelligent Technologies (CONIT). Hubli, India: IEEE. pp.1–7. https://doi.org/10.1109/CONIT51480.2021.9498383.
GUPTA, A., D’CUNHA, A., AWASTHI, K. AND BALASUBRAMANIAN, V., 2016. DAiSEE: Towards User Engagement Recognition in the Wild. https://doi.org/10.48550/arXiv.1609.01885.
KRITHIKA L.B AND LAKSHMI PRIYA GG, 2016. Student Emotion Recognition System (SERS) for e-learning Improvement Based on Learner Concentration Metric. Procedia Computer Science, 85, pp.767–776. https://doi.org/10.1016/j.procs.2016.05.264.
LEK, J.X.-Y. AND TEO, J., 2023. Academic Emotion Classification Using FER: A Systematic Review. Human Behavior and Emerging Technologies, 2023, pp.1–27. https://doi.org/10.1155/2023/9790005.
LI, G., 2018. Special Treatment of Video Image Based on FFmpeg. In: Proceedings of the 2018 Joint International Advanced Engineering and Technology Research Conference (JIAET 2018). [online] 2018 Joint International Advanced Engineering and Technology Research Conference (JIAET 2018). Xi’an, China: Atlantis Press. https://doi.org/10.2991/jiaet-18.2018.47.
NURALIF, I., YUNIARNO, E.M., SUPRAPTO, Y.K. AND WICAKSONO, A.A., 2023. Driver Fatigue Detection Based On Face Mesh Features Using Deep Learning. In: 2023 International Seminar on Intelligent Technology and Its Applications (ISITIA). [online] 2023 International Seminar on Intelligent Technology and Its Applications (ISITIA). Surabaya, Indonesia: IEEE. pp.1–5. https://doi.org/10.1109/ISITIA59021.2023.10221053.
NURRAHMA ROSANTI PAIDJA, A. AND BACHTIAR, F.A., 2022. Engagement Emotion Classification through Facial Landmark Using Convolutional Neural Network. In: 2022 2nd International Conference on Information Technology and Education (ICIT&E). [online] 2022 2nd International Conference on Information Technology and Education (ICIT&E). Malang, Indonesia: IEEE. pp.234–239. https://doi.org/10.1109/ICITE54466.2022.9759546.
PEDREGOSA, F., VAROQUAUX, G., GRAMFORT, A., MICHEL, V., THIRION, B., GRISEL, O., BLONDEL, M., PRETTENHOFER, P., WEISS, R., DUBOURG, V., VANDERPLAS, J., PASSOS, A. AND COURNAPEAU, D., n.d. Scikit-learn: Machine Learning in Python. MACHINE LEARNING IN PYTHON.
SIAM, A.I., SOLIMAN, N.F., ALGARNI, A.D., ABD EL-SAMIE, F.E. AND SEDIK, A., 2022. Deploying Machine Learning Techniques for Human Emotion Detection. Computational Intelligence and Neuroscience, 2022, pp.1–16. https://doi.org/10.1155/2022/8032673.
SOLANKI, N. AND MANDAL, S., 2022. Engagement Analysis Using DAiSEE Dataset. In: 2022 17th International Conference on Control, Automation, Robotics and Vision (ICARCV). [online] 2022 17th International Conference on Control, Automation, Robotics and Vision (ICARCV). Singapore, Singapore: IEEE. pp.223–228. https://doi.org/10.1109/ICARCV57592.2022.10004250.
SUN, Y., WONG, A.K.C. AND KAMEL, M.S., 2009. CLASSIFICATION OF IMBALANCED DATA: A REVIEW. International Journal of Pattern Recognition and Artificial Intelligence, 23(04), pp.687–719. https://doi.org/10.1142/S0218001409007326.
THUSEETHAN, S., RAJASEGARAR, S. AND YEARWOOD, J., 2019. Detecting Micro-expression Intensity Changes from Videos Based on Hybrid Deep CNN. In: Q. Yang, Z.-H. Zhou, Z. Gong, M.-L. Zhang and S.-J. Huang, eds. Advances in Knowledge Discovery and Data Mining, Lecture Notes in Computer Science. [online] Cham: Springer International Publishing. pp.387–399. https://doi.org/10.1007/978-3-030-16142-2_30.
WHITEHILL, J., SERPELL, Z., LIN, Y.-C., FOSTER, A. AND MOVELLAN, J.R., 2014. The Faces of Engagement: Automatic Recognition of Student Engagementfrom Facial Expressions. IEEE Transactions on Affective Computing, 5(1), pp.86–98. https://doi.org/10.1109/TAFFC.2014.2316163.
YUN, H., 2021. Prediction model of algal blooms using logistic regression and confusion matrix. International Journal of Electrical and Computer Engineering (IJECE), 11(3), p.2407. https://doi.org/10.11591/ijece.v11i3.pp2407-2413.
Unduhan
Diterbitkan
Terbitan
Bagian
Lisensi
Hak Cipta (c) 2025 Jurnal Teknologi Informasi dan Ilmu Komputer
Artikel ini berlisensiCreative Commons Attribution-ShareAlike 4.0 International License.
Artikel ini berlisensi Creative Common Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)
Penulis yang menerbitkan di jurnal ini menyetujui ketentuan berikut:
- Penulis menyimpan hak cipta dan memberikan jurnal hak penerbitan pertama naskah secara simultan dengan lisensi di bawah Creative Common Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) yang mengizinkan orang lain untuk berbagi pekerjaan dengan sebuah pernyataan kepenulisan pekerjaan dan penerbitan awal di jurnal ini.
- Penulis bisa memasukkan ke dalam penyusunan kontraktual tambahan terpisah untuk distribusi non ekslusif versi kaya terbitan jurnal (contoh: mempostingnya ke repositori institusional atau menerbitkannya dalam sebuah buku), dengan pengakuan penerbitan awalnya di jurnal ini.
- Penulis diizinkan dan didorong untuk mem-posting karya mereka online (contoh: di repositori institusional atau di website mereka) sebelum dan selama proses penyerahan, karena dapat mengarahkan ke pertukaran produktif, seperti halnya sitiran yang lebih awal dan lebih hebat dari karya yang diterbitkan. (Lihat Efek Akses Terbuka).
