Klasifikasi Laporan Keluhan Pelayanan Publik Berdasarkan Instansi Menggunakan Metode LDA-SVM

Penulis

Muhammad Alkaff, Andreyan Rizky Baskara, Irham Maulani

Abstrak

Sebuah sistem layanan untuk menyampaikan aspirasi dan keluhan masyarakat terhadap layanan pemerintah Indonesia, bernama Lapor! Pemerintah sudah lama memanfaatkan sistem tersebut untuk menjawab permasalahan masyarakat Indonesia terkait permasalahan birokrasi. Namun, peningkatan volume laporan dan pemilahan laporan yang dilakukan oleh operator dengan membaca setiap keluhan yang masuk melalui sistem menyebabkan sering terjadi kesalahan dimana operator meneruskan laporan tersebut ke instansi yang salah. Oleh karena itu, diperlukan suatu solusi yang dapat menentukan konteks laporan secara otomatis dengan menggunakan teknik Natural Language Processing. Penelitian ini bertujuan untuk membangun klasifikasi laporan secara otomatis berdasarkan topik laporan yang ditujukan kepada instansi yang berwenang dengan menggabungkan metode Latent Dirichlet Allocation (LDA) dan Support Vector Machine (SVM). Proses pemodelan topik untuk setiap laporan dilakukan dengan menggunakan metode LDA. Metode ini mengekstrak laporan untuk menemukan pola tertentu dalam dokumen yang akan menghasilkan keluaran dalam nilai distribusi topik. Selanjutnya, proses klasifikasi untuk menentukan laporan agensi tujuan dilakukan dengan menggunakan SVM berdasarkan nilai topik yang diekstraksi dengan metode LDA. Performa model LDA-SVM diukur dengan menggunakan confusion matrix dengan menghitung nilai akurasi, presisi, recall, dan F1 Score. Hasil pengujian menggunakan teknik split train-test dengan skor 70:30 menunjukkan bahwa model menghasilkan kinerja yang baik dengan akurasi 79,85%, presisi 79,98%, recall 72,37%, dan Skor F1 74,67%.

 

Abstract

A service system to convey aspirations and complaints from the public against Indonesia's government services, named Lapor! The Government has used the Government for a long time to answer the problems of the Indonesian people related to bureaucratic problems. However, the increasing volume of reports and the sorting of reports carried out by operators by reading every complaint that comes through the system cause frequent errors where operators forward the reports to the wrong agencies. Therefore, we need a solution that can automatically determine the report's context using Natural Language Processing techniques. This study aims to build automatic report classifications based on report topics addressed to authorized agencies by combining Latent Dirichlet Allocation (LDA) and Support Vector Machine (SVM). The topic-modeling process for each report was carried out using the LDA method. This method extracts reports to find specific patterns in documents that will produce output in topic distribution values. Furthermore, the classification process to determine the report's destination agency carried out using the SVM based on the value of the topics extracted by the LDA method. The LDA-SVM model's performance is measured using a confusion matrix by calculating the value of accuracy, precision, recall, and F1 Score. The test results using the train-test split technique with a 70:30 show that the model produces good performance with 79.85% accuracy, 79.98% precision, 72.37% recall, and 74.67% F1 Score


Teks Lengkap:

PDF

Referensi


ARUN, R., SURESH, V., MADHAVAN, C. E. V. & MURTY, M. N., 2010. On Finding the Natural Number of Topics with. Berlin, s.n.

ASUNCION, A., WELLING, M., SMYTH, P. & TEH, Y. W., 2009. On Smoothing and Inference for Topic Models. Montreal , s.n.

BARDENET, R., BRENDEL, M., KEGL, B. & SEBAG, M., 2013. Collaborative hyperparameter tuning. Atlanta, s.n.

BELEITES, C. dkk., 2012. Sample Size Planning for Classification Models. Analytica Chimica Acta, Volume 760, pp. 25-33.

BLEI, M. D., NG, Y. A. & JORDAN, I. M., 2003. Latent Dirichlet Allocation. Journal of Machine Learning Research, Volume 3, pp. 993-1022.

CLAESEN, M. & MOOR, B. D., 2015. Hyperparameter Search in Machine Learning. Agadir, s.n.

CUI, L. dkk., 2014. A Hierarchy Method Based on LDA and SVM for News Classification. IEEE International Conference on Data Mining Workshop, Volume 2015, pp. 60-64.

DENNY, M. J. & SPIRLING, A., 2018. Text Preprocessing For Unsupervised Learning: Why It. Political Analysis, 26(2), pp. 168-189.

DING, J. & JIN, W., 2019. A Prior Setting that Improves LDA in both Document Representation and Topic Extraction. Budapest, s.n.

GEORGE, M., SOUNDARABAI, B. & KRISHNAMURTHI, K., 2017. Impact of Topic Modelling Methods and Text Classification Techniques in Text Mining: a Survey. International Journal of Advances in Electronics and Computer Science, 4(3).

GOJAREA, S., JOSHI, R. & GAIGAWARE, D., 2015. Analysis and Design of Selenium WebDriver Automation Testing. 2nd International Symposium on Big Data and Cloud Computing (ISBCC’15), Volume 50, pp. 341-346.

HARRINGTON, A. N., 2020. Hands-on Python Tutorial. Chicago: Loyola University of Chicago.

HOFFMAN, M. D. & BLEI, D. M., 2010. Online Learning for Latent Dirichlet Allocation. Advances in Neural Information Processing Systems, Volume 23, pp. 856-864.

JAMES, G., WITTEN, D., HASTIE, T. & TIBSHIRANI, R., 2017. An Introduction to Statistical Learnin with Applications in R. New York: Springer Science+Business.

KAUR, G. & KAUR, E. P., 2017. Novel Approach of Text Classification by SVM-RBF Kernel and Linear SVC. International journal of Advance Research, Ideas, and Inovations in Technology, 3(3).

KAUR, J. & BUTTAR, P. K., 2016. A Systematic Review on Stopword Removal Algorithms. International Journal on Future Revolution in Computer Science & Communication Engineering, 150(4), pp. 15-17.

Kementerian Pendayagunaan Aparatur Negara dan Reformasi Birokrasi, n.d. Lapor!. [Online]

Available at: https://www.lapor.go.id/

[Accessed 29 11 2019].

Kementerian Pendayagunaan Aparatur Negara dan Reformasi Birokrasi, n.d. Tentang LAPOR!. [Online]

Available at: https://www.lapor.go.id/tentang

[Accessed 29 11 2019].

KUHN, M. & JOHNSON, K., 2013. Over-Fitting and Model Tuning. In: Applied Predictive Modeling. New York: Springer Science+Business, pp. 69-71.

KUSUMANINGRUM, A. P., 2017. Optimization Of Support Vector Machine Parameters Using Genetic Algorithm For Microarray Data Classification. s.l., s.n.

KUSUMANINGRUM, R., WIEDJAYANTO, M., ADHY, S. & SURYONO, S., 2016. Classification of Indonesian News Articles based on Latent Dirichlet. in Proceedings of the 2016 International Conference on Data and Software Engineering (ICoDSE).

LI, K. dkk., 2011. Multi-class text categorization based on LDA and SVM. Procedia Engineering, Volume 15, pp. 1963-1967.

LOUKAS, D. K., 2016. Learning Scrapy. Birmingham: Packt Publishing.

MANTOVANI, R. G. dkk., 2015. Effectiveness of Random Search in SVM hyper-parameter tuning. Killarney, s.n.

MARYAM, N. S., 2016. Mewujudkan Good Governance Melalui Pelayanan Publik. Jurnal Ilmu Politik dan Komunikasi, VI(1), p. 2.

MEGAWATI, C., 2015. Analisis Aspirasi dan Pengaduan Di Situs Lapor! dengan Menggunakan Text Mining. pp. 1-3.

OBSORNE, D. & GAEBLER, T., 1992. Dalam: Reinventing Government. California: Addison-Wesley Publ. Co.,, p. 24.

PEDREGOSA, F. dkk., 2011. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, Volume 12, pp. 2825-2830.

SAGGAF, S., SAID, M. M. & SAGGAF, W. S., 2018. Dalam: Reformasi Pelayanan Publik di Negara Berkembang. s.l.:SAH MEDIA, pp. 26-27.

SARKAR, S. dkk., 2019. Application of Optimized Machine Learning Techniques for Prediction of Occupational Accidents. Computers and Operations Research, Volume 106, pp. 210-224.

SHESHASAAYEE, A. & THAILAMBAL, G., 2017. Comparison of Classification Algorithms in Text Mining. International Journal of Pure and Applied Mathematics, 116(22), pp. 425-433.

SLAMET, C. dkk., 2018. Web Scraping and Naïve Bayes Classification for Job Search Engine. IOP Conference Series: Materials Science and Engineering, Volume 288.

SYARIF, I., WILLS, G. & BENNET, A. P., 2016. SVM Parameter Optimization using Grid Search and Genetic Algorithm to Improve Classification Performance. Telecommunication Computing Electronics and Control (TELKOMNIKA), 14(4), pp. 1502-1509.

TALA, F. Z., 2003. A Study of Stemming Effects on Information Retrieval in Bahasa Indonesia.

TEH, Y. W., NEWMAN, D. & WELLING, M., 2007. A Collapsed Variational Bayesian Inference. Vancouver, s.n.

VABALAS, A., GOWEN, E., POLIAKOFF, E. & CASSON, A. J., 2019. Machine learning algorithm validation with a Limited Size Sample. Plos Name, 14(11).

WALLACH, H. M., MIMNO, D. & MCCALLUM, A., 2009. Rethinking LDA: Why Priors Matter. Vancouver, s.n.

WANG, B. & GONG, N. Z., 2018. Stealing Hyperparameters in Machine Learning. IEEE Symposium on Security and Privacy, pp. 36-52.

YOUNG, T., HAZARIKA, D., PORIA, S. & CAMBRIA, E., 2018. Recent Trends in Deep Learning Based Natural Language Processing. IEEE Computational Intelligence Magazine, 13(3), pp. 55-75.




DOI: http://dx.doi.org/10.25126/jtiik.2021863768