Aspect-Based Sentiment Analysis on User Perceptions of OVO using Latent Dirichlet Allocation and Support Vector Machine

Abstract

The rapid development of digital technology and the Internet has significantly influenced financial services in Indonesia, leading to the widespread use of digital wallets. One of the most prominent digital wallet platforms is OVO, which has received millions of user reviews across application stores. This study applies aspect-based sentiment analysis to better understand user perceptions from reviews of the OVO application (versions 3.115 to 3.119). A total of 17.086 reviews were collected through web scraping and refined to 4.996 relevant entries. Topic modeling using Latent Dirichlet Allocation (LDA) identified four main aspects frequently discussed by users: Transaction Efficiency, User Experience, Account Access and Registration, and Balance and Charges. However, automatic aspect labeling using LDA keywords achieved only 11.46% agreement with manual annotations, increasing to 40.60% after keyword refinement. Therefore, manual aspect annotation was adopted as the basis for sentiment labeling. Sentiment labeling was conducted by three annotators based on structured guidelines, achieving a Fleiss’ Kappa score of 0.9915. A classification model was then developed using the Support Vector Machine (SVM) algorithm across six testing scenarios. The best-performing model, using a Linear kernel without ML-SMOTE, achieved a macro-average precision of 0.843, recall of 0.786, and F1-Score of 0.804. These results demonstrate the model’s effectiveness in handling multi-label classification under imbalanced data conditions, particularly for well-distributed aspects such as Transaction Efficiency and User Experience, while highlighting challenges in minority-class detection for aspects such as Account Access and Registration and Balance and Charges.

Keywords

Aspect-based sentiment analysis; OVO; Latent Dirichlet Allocation; Support Vector Machine.

Full Text:

PDF

References

Y. E. Rachmad, A. A. Bakri, R. Nuraini, and T. W. Nurdiani, “Application of the Unified Theory of Acceptance and Use of Technology Method to Analyze Factors Influencing the Use of Digital Wallets in Indonesia,” J. Informasi dan Teknologi, pp. 229–234, 2024. https://dx.doi.org/10.60083/jidt.v6i1.504

T. W. Nurdiani, “Integrating marketing and finance to increase company performance in VUCA world: a case study on banking state-owned enterprise in Indonesia (MANDIRI, BRI, BTN, BNI),” European Journal of Business and Innovation Research, vol. 9, no. 5, pp. 27–32, 2021.

Y. Soelasih, “The factors of millennials’ continuance intention to use digital wallets in Indonesia,” Binus Business Review, vol. 13, no. 3, pp. 315–323, 2022. https://dx.doi.org/10.21512/bbr.v13i3.8561

D. Safitri and M. B. Nainggolan, “Implementation of campaign strategy for national non cash movement from bank of Indonesia,” in Proc. 3rd Int. Conf. on Transformation in Communications (IcoTiC 2017), Nov. 2017, pp. 13–17. Atlantis Press. https://dx.doi.org/10.2991/icotic-17.2017.3

R. Widjojo, “The development of digital payment systems in Indonesia: a review of go-pay and ovo e-wallets,” Economic Alternatives, no. 3, pp. 384–395, 2020. https://dx.doi.org/10.37075/EA.2020.3.03

A. Ciptarianto and Y. Anggoro, “E-Wallet application penetration for financial inclusion in Indonesia,” Int. J. Curr. Sci. Res. Rev., vol. 5, no. 2, pp. 319–332, 2022.

I. K. Wati, A. M. Soma, and H. Ispriyahadi, “What Influences User Preferences in Digital Payment Systems? (A Comparative Analysis of E-Wallet in Indonesia),” Int. J. Entrepreneurship, Business and Creative Economy, vol. 4, no. 1, p. 78, 2024. https://doi.org/10.31098/ijebce.v4i1.2033

S. D. Widiyaningsih and A. Pertiwi, “Analysis of OVO Application Sentiment Using Lexicon Based Method and K-Nearest Neighbor,” J. Ilm. Ekon. Bisnis, vol. 25, no. 1, pp. 14–28, 2020.

Y. Noh, S. Park, and S. B. Park, “Aspect-based sentiment analysis using aspect map,” Applied Sciences, vol. 9, no. 16, p. 3239, 2019. https://dx.doi.org/10.3390/app9163239

Y. T. Lutfi, M. Saputra, and R. Y. Fa’rifah, “Aspect-Based Sentiment Analysis in Identifying Factors Causing Technostress in Fintech Users Using Naïve Bayes Algorithm,” in Proc. Int. Conf. on Enterprise and Industrial Systems (ICOEINS 2023), Dec. 2023, pp. 107–117.

M. D. Pratama, R. Sarno, and R. Abdullah, “Sentiment Analysis User Regarding Hotel Reviews by Aspect Based Using Latent Dirichlet Allocation, Semantic Similarity, and Support Vector Machine Method,” Int. J. Intell. Eng. Syst., vol. 15, no. 3, 2022. https://doi.org/10.22266/ijies2022.0630.43

N. Hidayati, F. Hamami, and R. Y. Fa’rifah, “Aspect-Based Sentiment Analysis on FLIP Application Reviews (Play Store) Using Support Vector Machine (SVM) Algorithm,” J. Inform. Telecommun. Eng., vol. 7, no. 1, pp. 183–197, 2023.

H. Mustakim and S. Priyanta, “Aspect-Based Sentiment Analysis of KAI Access Reviews Using NBC and SVM,” IJCCS (Indones. J. Comput. Cybern. Syst.), vol. 16, no. 2, pp. 113–124, 2022. https://dx.doi.org/10.22146/ijccs.68903

W. A. Awadh, R. B. Sulaiman, and M. A. Mahmoud, “Aspect-based sentiment analysis in MOOCs: a systematic literature review introducing the MASC-MEF framework,” Journal of King Saud University – Computer and Information Sciences, vol. 37, no. 1, pp. 1–33, 2025. https://dx.doi.org/10.1007/s44443-025-00018-1

Y. Kalepalli, S. Tasneem, P. D. P. Teja, and S. Manne, “Effective comparison of LDA with LSA for topic modelling,” in Proc. 2020 4th Int. Conf. on Intelligent Computing and Control Systems (ICICCS), May 2020, pp. 1245–1250. IEEE.

R. Feldman and J. Sanger, The Text Mining Handbook: Advanced Approaches in Analyzing Unstructured Data. Cambridge: Cambridge University Press, 2007.

D. Irawan, E. B. Perkasa, Y. Yurindra, D. Wahyuningsih, and E. Helmud, “Perbandingan Klasifikasi SMS Berbasis Support Vector Machine, Naive Bayes Classifier, Random Forest dan Bagging Classifier [Comparison of SMS Classification Based on Support Vector Machine, Naive Bayes Classifier, Random Forest, and Bagging Classifier],” Jurnal Sisfokom (Sistem Informasi dan Komputer), vol. 10, no. 3, pp. 432–437, 2021. (in Indonesian).

F. Ghasemi and S. Sharifi, “Heart Failure Prediction Using Support Vector Machine,” International Journal of Novel Research in Life Sciences, vol. 12, no. 1, pp. 1–8, 2025.

A. Tabassum and R. R. Patil, “A survey on text pre-processing & feature extraction techniques in natural language processing,” Int. Res. J. Eng. Technol. (IRJET), vol. 7, no. 6, pp. 4864–4867, 2020.

B. Pahwa, S. Taruna, and N. Kasliwal, “Sentiment analysis-strategy for text pre-processing,” Int. J. Comput. Appl., vol. 180, no. 34, pp. 15–18, 2018. https://doi.org/10.5120/ijca2018916865

A. R. Royyan and E. B. Setiawan, “Feature expansion Word2Vec for sentiment analysis of public policy in Twitter,” J. RESTI (Rekayasa Sistem dan Teknologi Informasi), vol. 6, no. 1, pp. 78–84, 2022. https://doi.org/10.29207/resti.v6i1.3525

S. E. Saad and J. Yang, “Twitter sentiment analysis based on ordinal regression,” IEEE Access, vol. 7, pp. 163677–163685, 2019. https://dx.doi.org/10.1109/ACCESS.2019.2958804

Z. Yang, X. Wang, M. Qiu, S. Hou, and Y. Wu, “Account of Spatio-Temporal Characteristics in Customs Anti-Smuggling Intelligence Acquisition: A Combined LSTM+ CRF Model Using TF-IDF and Levenshtein,” International Journal of Data Warehousing and Mining (IJDWM), vol. 20, no. 1, pp. 1–20, 2024.

E. Chan, “The Most Optimal Machine Learning Model for Defense Stock Prediction,” Curieux Academic Journal, vol. 3, no. 2, pp. 45–56, 2024.

D. Krstinić, M. Braović, L. Šerić, and D. Božić-Štulić, “Multi-label classifier performance evaluation with confusion matrix,” Computer Science & Information Technology, vol. 1, pp. 1–14, 2020.

http://dx.doi.org/10.28989/avitec.v7i2.3035

Article Metrics

Abstract view: 65 times
Download     : 42   times

This work is licensed under a Creative Commons Attribution 4.0 International License.

Refbacks

• There are currently no refbacks.