AI-Aware Assessment of English Projects in Vocational Higher Education: A Conceptual Framework
Abstract
Generative artificial intelligence (GenAI) complicates judgments about individual English capability in collaborative multimodal projects, where group contributions, production quality, and AI assistance can obscure performance. This conceptual paper proposes an AI-aware assessment framework for a general English course serving 17 study programs in vocational higher education. A conceptual synthesis of literature on project-based language learning, multimodal composing, authentic assessment, collaborative learning, and GenAI assessment informs six principles: visible process, local specificity, multimodal and oral performance, individual accountability, declared and calibrated AI use, and reflective metacognition. The principles are illustrated through two existing course projects, including an alternative format for Project 2. Proposed adaptations appear in a principle-to-practice matrix and an illustrative assessment weighting scheme. The framework offers guidance for collecting complementary evidence of individual language capability while retaining vocationally relevant projects and shared language expectations. Its effectiveness and feasibility require empirical evaluation
Keywords
References
Bannert, M., Kasneci, E., Sessler, K., Stefan, K., Fischer, F., Gasser, U., Groh, G., Stephan, G., Poquet, O., Sailer, M., Schmidt, A., & Seidel, T. (2023). ChatGPT for Good ? On Opportunities and Challenges of Large Language Models for Education. 1–13.
Barrot, J. S. (2023). Using ChatGPT for second language writing: Pitfalls and potentials. Assessing Writing, 57, 100745. https://doi.org/10.1016/j.asw.2023.100745
Basturkmen, H. (2010). Developing courses in English for specific purposes. Palgrave Macmillan.
Beckett, G. H., & Slater, T. (2005). The project framework: A tool for language, content, and skills integration. ELT Journal, 59(2), 108–116. https://doi.org/10.1093/eltj/cci024
Bloxham, S., Hughes, C., & Adie, L. (2016). What’s the point of moderation? A discussion of the purposes achieved through contemporary moderation practices. Assessment & Evaluation in Higher Education, 41(4), 638–653. https://doi.org/10.1080/02602938.2015.1039932
Chan, C. K. Y. (2023). A comprehensive AI policy education framework for university teaching and learning. International Journal of Educational Technology in Higher Education, 20, 38. https://doi.org/10.1186/s41239-023-00408-3
Corbin, T., Dawson, P., & Liu, D. (2025). Talk is cheap: Why structural assessment changes are needed for a time of GenAI. Assessment & Evaluation in Higher Education, 50(7), 1087–1097. https://doi.org/10.1080/02602938.2025.2503964
Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2024). Chatting and Cheating: Ensuring Academic Integrity in the Era of ChatGPT. Innovations in Education and Teaching International, 61(2), 228–239. https://doi.org/10.1080/14703297.2023.2190148
Dawson, P., Bearman, M., Dollinger, M., & Boud, D. (2024). Validity matters more than cheating. Assessment & Evaluation in Higher Education, 49(7), 1005–1016. https://doi.org/10.1080/02602938.2024.2386662
Dudley-Evans, T., & St. John, M. J. (1998). Developments in English for Specific Purposes: A Multi-Disciplinary Approach. Cambridge University Press.
Gulikers, J. T. M., Bastiaens, T. J., & Kirschner, P. A. (2004). A five-dimensional framework for authentic assessment. Educational Technology Research and Development, 52(3), 67–86. https://doi.org/10.1007/BF02504676
Hafner, C A. (2014). Embedding digital literacies in English language teaching: Students’ digital video projects as multimodal ensembles. TESOL Quarterly, 48(4), 655–685. https://doi.org/10.1002/tesq.138
Hafner, C A, & Ho, W. Y. J. (2020). Assessing digital multimodal composing in second language writing: Towards a process-based model. Journal of Second Language Writing, 47, 100710. https://doi.org/10.1016/j.jslw.2020.100710
Hafner, Christoph A, & Miller, L. (2011). FOSTERING LEARNER AUTONOMY IN ENGLISH FOR SCIENCE : A COLLABORATIVE DIGITAL VIDEO PROJECT IN A TECHNOLOGICAL LEARNING ENVIRONMENT 1. 15(3), 68–86.
Hyland, K. (2007). Genre pedagogy: Language, literacy and L2 writing instruction. Journal of Second Language Writing, 16(3), 148–164. https://doi.org/10.1016/j.jslw.2007.07.005
Ion, G., Díaz-Vicario, A., & Mercader, C. (2024). Making steps towards improved fairness in group work assessment: The role of students’ self- and peer-assessment. Active Learning in Higher Education, 25(3), 425–437. https://doi.org/10.1177/14697874231154826
Jaakkola, E. (2020). Designing conceptual articles: Four approaches. AMS Review, 10(1–2), 18–26. https://doi.org/10.1007/s13162-020-00161-0
Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73. https://doi.org/10.1111/jedm.12000
Kemendikbudristek. (2021). Keputusan Menteri Pendidikan dan Kebudayaan Nomor 3/M/2021 tentang indikator kinerja utama perguruan tinggi negeri dan lembaga layanan pendidikan tinggi. Kementerian Pendidikan dan Kebudayaan Republik Indonesia.
Kemendikbudristek. (2024). Panduan penggunaan generative artificial intelligence (GenAI) pada pembelajaran di perguruan tinggi. Kementerian Pendidikan, Kebudayaan, Riset, dan Teknologi Republik Indonesia.
Kress, G. (2010). Multimodality: A social semiotic approach to contemporary communication. Routledge.
Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023). GPT detectors are biased against non-native English writers. Patterns, 4(7), 100779. https://doi.org/10.1016/j.patter.2023.100779
Lodge, J. M., Howard, S., Bearman, M., & Dawson, P. (2023). Assessment reform for the age of artificial intelligence. Tertiary Education Quality and Standards Agency.
Meijer, H., Brouwer, J., Hoekstra, R., & Strijbos, J.-W. (2022). Exploring construct and consequential validity of collaborative learning assessment in higher education. Small Group Research, 53(6), 891–925. https://doi.org/10.1177/10464964221095545
Perkins, M., Furze, L., Roe, J., & MacVaugh, J. (2024). The Artificial Intelligence Assessment Scale (AIAS): A framework for ethical integration of generative AI in educational assessment. Journal of University Teaching and Learning Practice, 21(6). https://doi.org/10.53761/q3azde36
Perkins, M., Roe, J., Postma, D., McGaughran, J., & Hickerson, D. (2024). Detection of GPT-4 Generated Text in Higher Education: Combining Academic Judgement and Software to Identify Generative AI Tool Misuse. Journal of Academic Ethics, 22(1), 89–113. https://doi.org/10.1007/s10805-023-09492-6
Puteri, C. G., Cahyono, B. Y., Widiati, U., & Suryati, N. (2026). Using ChatGPT as a writing assistant: A study on essay quality development among Indonesian EFL university students. Indonesian Journal of Applied Linguistics, 15(2), 332–346. https://doi.org/10.17509/3ebmy158
Stoller, F. L. (2006). Establishing a theoretical foundation for project-based learning in second and foreign language contexts. In G. H. Beckett & P. C. Miller (Eds.), Project-based second and foreign language education: Past, present, and future (pp. 19–40). Information Age Publishing.
Tai, J., Ajjawi, R., Boud, D., Dawson, P., & Panadero, E. (2018). Developing evaluative judgement: Enabling students to make decisions about the quality of work. Higher Education, 76(3), 467–481. https://doi.org/10.1007/s10734-017-0220-3
Villarroel, V., Bloxham, S., Bruna, D., Bruna, C., & Herrera-Seda, C. (2018). Authentic assessment: Creating a blueprint for course design. Assessment & Evaluation in Higher Education, 43(5), 840–854. https://doi.org/10.1080/02602938.2017.1412396
Weber-Wulff, D., Anohina-Naumeca, A., Bjelobaba, S., Foltýnek, T., Guerrero-Dib, J., Popoola, O., Šigut, P., & Waddington, L. (2023). Testing of detection tools for AI-generated text. International Journal for Educational Integrity, 19, 26. https://doi.org/10.1007/s40979-023-00146-z
Xia, Q., Weng, X., Ouyang, F., Lin, T. J., & Chiu, T. K. F. (2024). A scoping review on how generative artificial intelligence transforms assessment in higher education. International Journal of Educational Technology in Higher Education, 21, 40. https://doi.org/10.1186/s41239-024-00468-z
DOI: https://doi.org/10.31004/jele.v11i5.3352
Refbacks
- There are currently no refbacks.
Copyright (c) 2026 Irma Rasita Gloria Barus, Bontisesari Bontisesari
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
