Development of a Chatbot-Based Assessment System for Evaluating Career Readiness in Vocational Education
DOI:
https://doi.org/10.33394/jp.v13i1.18781Keywords:
Chatbot, Career Readiness, Employability Skills, Vocational High School, BotpressAbstract
This study aims to develop, assess the feasibility of, and evaluate the effectiveness of a web-based chatbot, TemanKarir, for measuring vocational high school students’ career readiness. The research employed a Research and Development (R&D) method using the ADDIE development model, which consists of five stages: analysis, design, development, implementation, and evaluation. The chatbot was developed using the Botpress platform and integrated with Google Sheets, WordPress, the Telegram Bot API, and QuickChart. The assessment instrument was based on the Employability Skills Framework and comprised 75 items across nine skill dimensions. All items demonstrated strong content validity (Aiken’s V = 0.75–1.00), satisfactory item validity (r = 0.339–0.813), and high internal consistency (Cronbach’s α = 0.985). The participants included two content experts, two media experts, and 50 Grade XI vocational high school students. Data were analyzed using descriptive statistical techniques based on a four-point Likert scale and software quality evaluation criteria from the ISO/IEC 25010 standard. The results indicate that: (1) the TemanKarir chatbot was successfully developed with key features, including career readiness assessment, automated feedback, result summaries for guidance counselors, and follow-up guidance recommendations; (2) the chatbot was rated as highly feasible based on evaluations by content experts (mean score = 3.62) and media experts (mean score = 3.52); and (3) the chatbot demonstrated high effectiveness as an assessment tool, with strong scores for effectiveness (3.57), efficiency (3.58), and user satisfaction (3.56). Overall, the findings confirm that the TemanKarir chatbot is both feasible and effective for assessing vocational high school students’ career readiness and contributes positively to their career development processes.
References
Almanasreh, E., Moles, R., & Chen, T. F. (2019). Evaluation of methods used for estimating content validity. Research in Social and Administrative Pharmacy, 15(2), 214–221. https://doi.org/10.1016/j.sapharm.2018.03.066
Areisy, H. J., & Sudira, P. (2022). Employability Skills of State Vocational High School Students on Welding Engineering Expertise Competency. Jurnal Pendidikan Vokasi, 12(1), 53–63. https://doi.org/10.21831/jpv.v12i1.41764
Badan Pusat Statistik. (2023). Statistik ketenagakerjaan Kabupaten Dompu. BPS Kabupaten Dompu.
Badan Pusat Statistik. (2024). Keadaan Ketenagakerjaan Indonesia Februari 2024. Berita Statistik Indonesia, 05(28), 1–28. http://www.bps.go.id/brs_file/tenaker-15mei07.pdf
Branch, R. M. (2010). Instructional design: The ADDIE approach. In Instructional Design: The ADDIE Approach. https://doi.org/10.1007/978-0-387-09506-6
Dalvi, V., Choudhary, C., Kokitkar, V., Kulkarni, S., & Potdar, G. (2024). Chatbot for Academic / Career Guidance. Computer Research and Development, 24(1000), 73–78.
DEST. (2004). Employability Skills : Final Report : Development of A Strategy to Support The Universal Recognition and Recording of Employability Skills ; A Skills Portfolio Approach. December, 76.
Dongre, G., Vadraiwala, F., Upadhyaya, J., & Ghosal, D. (2021). Web Based Career Counselling with ChatBot (V-Counselling). International Research Journal of Engineering and Technology, 8(4), 1739–1743. www.irjet.net
Fleischmann, R., van Aken, D., Bengel, J., & Klein, C. (2017). Electronic and paper-based data collection in clinical trials: A comparison of methods. BMC Medical Research Methodology, 17(1), 136.
Garza, M. Y., Quinn, S. C., Li, Y., & Butler, J. C. (2023). Error rates of data processing methods: Implications for data quality and timeliness. JMIR Medical Informatics.
Hamidah, T., Putri, O. F., Kurniawan, T., Puspitasari, E. I., Khotimah, K., & Suryawati, C. T. (2022). Problematika Bimbingan dan Konseling Bidang Karir Siswa SMK: A Systematic Literature Review (SLR). TERAPUTIK: Jurnal Bimbingan Dan Konseling, 5(3), 294–304. https://doi.org/10.26539/teraputik.53852
Hamzati, N., & Naqiyah, N. (2023). Bimbingan Karier dengan Teknik Modeling pada Peserta Didik Sekolah Menengah : Systematic Literature Review. Indonesian Journal of Guidance and Counseling: Theory and Application, 12(1), 1–17.
Hasdiana, Hadjaratie, L., Masaong, Abd. K., & Panai, A. H. (2023). Mempersiapkan Generasi Emas Indonesia 2045 Menghadapi Bonus Demografi Melalui Pembelajaran Berbasis Kreatifitas. AKSARA: Jurnal Ilmu Pendidikan Nonformal, 09(2), 949–958. http://ejurnal.pps.ung.ac.id/index.php/Aksara
Hidayati, A., Dzil Barr, F., & Sigit, K. N. (2021). Ekuitas: Jurnal Pendidikan Ekonomi Kesesuaian Kompetensi Lulusan SMK dengan Kebutuhan Dunia Usaha dan Industri. Ekuitas: Jurnal Pendidikan Ekonomi, 9(2), 284–292. https://ejournal.undiksha.ac.id/index.php/EKU
Lee, T., Jagannath, K., Anggarwal, N., Sridar, R., Wilde, S., Hill, T., & Chen, Y. (2019). Intelligent Career Advisers in Your Pocket ? Twenty-Fifth Americas Conference on Information Systems, Cancun, 1–5.
Mhatre, M. S., Chaudhari, K. A., Jadhav, A. V., & Sathe, V. A. (2024). Edu Counselor: Career Counseling Chatbot. International Journal of Engineering Applied Sciences and Technology, 8(10), 104–108. http://www.ijeast.com
Nabilla, G. D., Suhandiah, S., & Supriyanto, A. (2025). Generation Z’s Work Readiness in the Digital Era: Exploring the Role of Internship Experience and Digital Literacy, with Psychological Capital as a Mediator. ICONBIT, 1088–1099.
Nu’ma, F. A., & Mangunsong, F. M. (2024). Psychological Capital of Generation Z in Facing The Workforce: A Narrative Literature Review. Konselor, 13(2), 108–120. https://doi.org/10.24036/0202413271-0-86
Pohan, M., & Rialdy, N. (2024). Analisis Faktor Yang Mempengaruhi Perilaku Gen Z Dalam Menentukan Bisnis Yang Diminati Sebagai Usaha Kecil. Jurnal Ilmiah Ekonomi Dan Manajemen, 2(6), 295–302. https://doi.org/10.61722/jiem.v2i6.1430
Ristianti, D., Sofyan, A., Aminah, S., & S Muryono. (2025). Challenges and Opportunities of Integrating AI in Guidance and Counseling Services for Students in Islamic Higher Education. Counselor, 14(1), 2025. https://counselor.ppj.unp.ac.id/index.php/konselor/article/view/99
Saripah, I., Priliani, D. R., & Nadhirah, N. A. (2023). Problematika Kematangan Karir Peserta Didik Sekolah Menengah Kejuruan : Implementasi Pada Layanan Bimbingan dan Konseling Karir. Indonesian Journal of Guidance and Counseling: Theory and Application, 12(1), 95–118.
Suresh, N., Mukabe, N., Hashiyana, V., Limbo, A., & Hauwanga, A. (2021). Career Counseling Chatbot on Facebook Messenger using AI. ACM International Conference Proceeding Series, 65–73. https://doi.org/10.1145/3484824.3484875
Thasfa, S. A., & Daulay, N. (2024). Upaya Guru Bimbingan dan Konseling Terhadap Kematangan Karir Remaja Melalui Bimbingan Karir. G-Couns: Jurnal Bimbingan Dan Konseling, 9(1), 223–232. https://doi.org/10.31316/gcouns.v9i1.6314
Tsaqib, A. F., Wiyono, A., & Rusimamto, P. W. (2025). Pengaruh Employability Skills Terhadap Kesiapan Kerja Siswa. LEARNING : Jurnal Inovasi Penelitian Pendidikan Dan Pembelajaran, 5(3), 1121–1130.
Walther, B., Hossin, S., Townend, J., Abernethy, N., Parker, D., & Jeffries, D. (2011). Comparison of electronic data capture (EDC) with the standard data capture method for clinical trial data. PLOS ONE, 6(9), e25348.
Published
How to Cite
Issue
Section
Citation Check
License
Copyright (c) 2026 The Author(s)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
License and Publishing Agreement
In submitting the manuscript to the journal, the authors certify that:
- They are authorized by their co-authors to enter into these arrangements.
- The work described has not been formally published before, except in the form of an abstract or as part of a published lecture, review, thesis, or overlay journal.
- That it is not under consideration for publication elsewhere,
- That its publication has been approved by all the author(s) and by the responsible authorities tacitly or explicitly of the institutes where the work has been carried out.
- They secure the right to reproduce any material that has already been published or copyrighted elsewhere.
- They agree to the following license and publishing agreement.
Copyright
Authors who publish with Jurnal Paedagogy agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License (CC BY-SA 4.0) that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
Licensing for Data Publication
-
Open Data Commons Attribution License, http://www.opendatacommons.org/licenses/by/1.0/ (default)
