Optimasi Klasifikasi Sentimen Komentar Pengguna Game Bergerak Menggunakan Svm, Grid Search dan Kombinasi N-Gram
DOI:
https://doi.org/10.25126/jtiik.1148244Abstrak
Game online telah menjadi fenomena budaya signifikan dalam industri yang berkembang pesat. Pengguna dan pengembang game menggunakan analisis sentimen untuk memahami opini dan ulasan pemain, yang membantu dalam pengembangan dan peningkatan game. Penelitian ini melakukan klasifikasi sentimen menggunakan algoritma Support Vector Machine (SVM) dengan penerapan teknik N-Gram untuk seleksi fitur. Grid Search (GS) digunakan untuk optimasi hyperparameter guna mencapai akurasi optimal. Eksperimen dilakukan dengan berbagai skenario, termasuk variasi jumlah data, pengaturan hyperparameter, rasio dataset pelatihan dan pengujian, serta konfigurasi N-Gram. Kinerja model dinilai menggunakan metrik seperti Akurasi, Presisi, Recall, dan Area di Bawah Kurva ROC (AUC). Hasil menunjukkan bahwa dengan dataset gabungan (Allgame) dan integrasi fitur seleksi N-Gram Unigram, Bigram, dan Trigram (UniBiTri), model ini mencapai akurasi 87,3%, presisi 88,5%, recall 85,5%, dan AUC 0,9081, menggunakan kernel Fungsi Basis Radial (RBF) dengan validasi silang k-fold (k=10).
Abstract
Online gaming has become a significant cultural phenomenon within a rapidly expanding industry. Game users and developers leverage sentiment analysis to understand player opinions and reviews, which subsequently guide game development and enhancements. In this study, sentiment classification was performed using the Support Vector Machine (SVM) algorithm, employing N-Gram techniques for feature selection. Grid Search (GS) was utilized for hyperparameter optimization to achieve the highest possible accuracy. To evaluate the impact of these methods, experiments were conducted across various scenarios, including different data quantities, hyperparameter settings, training and testing dataset ratios, and N-Gram configurations. The performance of the classification model was assessed using metrics such as Accuracy, Precision, Recall, and the Area Under the ROC Curve (AUC). The results of the study indicate that by using 3600 rows from a combined dataset (Allgame) and integrating Unigram, Bigram, and Trigram (UniBiTri) N-Gram selection features, along with k-fold cross-validation (k=10) and the Radial Basis Function (RBF) kernel, the model effectively classifies user reviews. Specifically, the model achieved an accuracy of 87.3%, precision of 88.5%, recall of 85.5%, and an AUC of 0.9081.
Downloads
Referensi
ABIMANYU, D., BUDIANITA, E., CYNTHIA, E.P., YANTO, F., YUSRA, Y., 2022. Analisis Sentimen Akun Twitter Apex Legends Menggunakan VADER. Jurnal Nasional Komputasi dan Teknologi Informasi (JNKTI) 5, 423–431. https://doi.org/10.32672/jnkti.v5i3.4382
ARIFIN, N., ENRI, U., SULISTIYOWATI, N., 2021. Penerapan Algoritma Support Vector Machine (SVM) dengan TF-IDF N-Gram untuk Text Classification. STRING 6, 129. https://doi.org/10.30998/string.v6i2.10133
BIRJALI, M., KASRI, M., BENI-HSSANE, A., 2021. A comprehensive survey on sentiment analysis: Approaches, challenges and trends. Knowledge-Based Systems 226, 107134. https://doi.org/10.1016/j.knosys.2021.107134
CHONG, K., SHAH, N., 2022. Comparison of Naive Bayes and SVM Classification in Grid-Search Hyperparameter Tuned and Non-Hyperparameter Tuned Healthcare Stock Market Sentiment Analysis.
IJACSA 13. https://doi.org/10.14569/IJACSA.2022.0131213
ELGELDAWI, E., SAYED, A., GALAL, A.R., ZAKI, A.M., 2021. Hyperparameter Tuning for Machine Learning Algorithms Used for Arabic Sentiment Analysis. Informatics 8, 79. https://doi.org/10.3390/informatics8040079
FEBRIANTA, M.Y., WIDIYANESTI, S., RAMADHAN, S.R., 2021. Analisis Ulasan Indie Video Game Lokal pada Steam Menggunakan Analisis Sentimen dan Pemodelan Topik Berbasis Latent Dirichlet Allocation. Journal of Animation and Games Studies 7, 117–144.
https://doi.org/10.24821/jags.v7i2.5162
FIDE, S., SUPARTI, S., SUDARNO, S., 2021. Analisis Sentimen Ulasan Aplikasi Tiktok Di Google Play Menggunakan Metode Support Vector Machine (SVM) dan Asosiasi. Jurnal Gaussian 10, 346–358. https://doi.org/10.14710/j.gauss.v10i3.32786
HENDRIYANTO, M.D., RIDHA, A.A., ENRI, U., 2022. Analisis Sentimen Ulasan Aplikasi Mola Pada Google Play Store Menggunakan Algoritma Support Vector Machine. INTECOMS: Journal of Information Technology and Computer Science 5, 1–7. https://doi.org/10.31539/intecoms.v5i1.3708
IRIANANDA, S.W., BUDIAWAN, R.W., RAHMAN, A.Y., ISTIADI, I., 2022. Kinerja Seleksi Fitur N-Gram Pada Analisis Sentimen Ulasan Mobile Game di Google Playstore. Conference on Innovation and Application of Science and Technology (CIASTECH) 5, 639–648.
IRIANANDA, S.W., PUTRA, R.P., FARHAN, A., 2023a. KINERJA AUTO LABELLING PADA Analisis Sentimen Terhadap Pasangan Calon Presiden 2024 di Media Sosial X. Conference on Innovation and Application of Science and Technology (CIASTECH) 6, 618–633. https://doi.org/10.31328/ciastech.v6i1.5354
IRIANANDA, S.W., PUTRA, R.P., NUGROHO, K.S., 2021. Analisis Sentimen dan Analisis Data Eksploratif Ulasan Aplikasi Marketplace Google Playstore. Presented at the Conference on Innovation and Application of Science and Technology (CIASTECH) 2021, Universitas Widyagama Malang, Malang, Indonesia, p. 10.
IRIANANDA, S.W., PUTRA, R.P., RAIHAN, A.A., SAPUTRA, D.A., VERDIANSYAH, E., 2023b. Analisis Sentimen Ulasan Game Mobile First-Person Shooter Di Google Play Store Menggunakan Metode Pembobotan TF-IDF. Prosidia Widya Saintek 2, 281–288.
KHALID, R., JAVAID, N., 2020. A survey on hyperparameters optimization algorithms of forecasting models in smart grid. Sustainable Cities and Society 61, 102275. https://doi.org/10.1016/j.scs.2020.102275
KING, D.L., DELFABBRO, P.H., BILLIEUX, J., POTENZA, M.N., 2020. Problematic online gaming and the COVID-19 pandemic. Journal of Behavioral Addictions 9, 184–186. https://doi.org/10.1556/2006.2020.00016
KUSNADI, R., YUSUF, Y., ANDRIANTONY, A., ARDIAN YAPUTRA, R., CAINTAN, M., 2021. Analisis Sentimen Terhadap Game Genshin Impact Menggunakan Bert. rabit 6, 122–129. https://doi.org/10.36341/rabit.v6i2.1765
LIANG, S., 2021. Comparative Analysis of SVM, XGBoost and Neural Network on Hate Speech Classification. RESTI 5, 896–903. https://doi.org/10.29207/resti.v5i5.3506
NUGRAHA, W., SASONGKO, A., 2022. Hyperparameter Tuning on Classification Algorithm with Grid Search. SISTEMASI 11, 391. https://doi.org/10.32520/stmsi.v11i2.1750
NURCAHYO, J.A., SASONGKO, T.B., 2023. Hyperparameter Tuning Algoritma Supervised Learning untuk Klasifikasi Keluarga Penerima Bantuan Pangan Beras. Indonesian Journal of Computer Science 12. https://doi.org/10.33022/ijcs.v12i3.3254
PAMUNGKAS, F.S., KHARISUDIN, I., 2021. Analisis Sentimen dengan SVM, Naive Bayes dan KNN untuk Studi Tanggapan Masyarakat Indonesia Terhadap Pandemi Covid-19 pada Media Sosial Twitter 4, 7.
PRABU, S., THIYANESWARAN, B., SUJATHA, M., NALINI, C., RAJKUMAR, S., 2022. Grid Search for Predicting Coronary Heart Disease by Tuning Hyper-Parameters. Computer Systems Science and Engineering 43, 737–749. https://doi.org/10.32604/csse.2022.022739
PRATMANTO, D., ROUSYATI, R., WATI, F.F., WIDODO, A.E., SULEMAN, S., WIJIANTO, R., 2020. App Review Sentiment Analysis Shopee Application In Google Play Store Using Naive Bayes Algorithm. J. Phys.: Conf. Ser. 1641, 012043. https://doi.org/10.1088/1742-6596/1641/1/012043
PUJADAYANTI, I., FAUZI, M.A., SARI, Y.A., 2018. Prediksi Rating Otomatis pada Ulasan Produk Kecantikan dengan Metode Naïve Bayes dan N-gram 7.
RADZI, S.F.M., KARIM, M.K.A., SARIPAN, M.I., RAHMAN, M.A.A., ISA, I.N.C., IBAHIM, M.J., 2021. Hyperparameter Tuning and Pipeline Optimization via Grid Search Method and Tree-Based AutoML in Breast Cancer Prediction. J Pers Med 11, 978. https://doi.org/10.3390/jpm11100978
RAHMAN, A., UTAMI, E., SUDARMAWAN, S., 2021. Sentimen Analisis Terhadap Aplikasi pada Google Playstore Menggunakan Algoritma Naïve Bayes dan Algoritma Genetika. JKKI 5, 60–71. https://doi.org/10.31603/komtika.v5i1.5188
SIDIQ, R.P., DERMAWAN, B.A., UMAIDAH, Y., 2020. Sentimen Analisis Komentar Toxic pada Grup Facebook Game Online Menggunakan Klasifikasi Naïve Bayes. JIUP 5, 356. https://doi.org/10.32493/informatika.v5i3.6571
SUJADI, H., 2022. Analisis Sentimen Pengguna Media Sosial Twitter Terhadap Wabah Covid-19 Dengan Metode Naive Bayes Classifier Dan Support Vector Machine. INFOTECH journal 8, 22–27. https://doi.org/10.31949/infotech.v8i1.1883
THENATA, A.P., 2021. Text Mining Literature Review on Indonesian Social Media. JEPIN 7, 226. https://doi.org/10.26418/jp.v7i2.47975
Unduhan
Diterbitkan
Terbitan
Bagian
Lisensi
Hak Cipta (c) 2024 Jurnal Teknologi Informasi dan Ilmu Komputer
Artikel ini berlisensiCreative Commons Attribution-ShareAlike 4.0 International License.
Artikel ini berlisensi Creative Common Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)
Penulis yang menerbitkan di jurnal ini menyetujui ketentuan berikut:
- Penulis menyimpan hak cipta dan memberikan jurnal hak penerbitan pertama naskah secara simultan dengan lisensi di bawah Creative Common Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) yang mengizinkan orang lain untuk berbagi pekerjaan dengan sebuah pernyataan kepenulisan pekerjaan dan penerbitan awal di jurnal ini.
- Penulis bisa memasukkan ke dalam penyusunan kontraktual tambahan terpisah untuk distribusi non ekslusif versi kaya terbitan jurnal (contoh: mempostingnya ke repositori institusional atau menerbitkannya dalam sebuah buku), dengan pengakuan penerbitan awalnya di jurnal ini.
- Penulis diizinkan dan didorong untuk mem-posting karya mereka online (contoh: di repositori institusional atau di website mereka) sebelum dan selama proses penyerahan, karena dapat mengarahkan ke pertukaran produktif, seperti halnya sitiran yang lebih awal dan lebih hebat dari karya yang diterbitkan. (Lihat Efek Akses Terbuka).
