Automated formative assessment with AI in higher education
DOI:
https://doi.org/10.62305/alcon.v5i4.782Keywords:
assessment; feedback; automation; rubrics; artificial intelligence; learningAbstract
Open-ended questions in formative assessment require timely, specific, and criteria-based feedback to be pedagogically effective, but their manual production is time-consuming and presents inter-rater variability. This study analyzes the implementation of an artificial intelligence-assisted formative assessment system deployed at the Universidad Estatal del Sur de Manabí (UNESUM), which automates the grading of open-ended questions using explicit analytic rubrics and generates personalized feedback at a course scale. The architecture integrates Google Forms for collection, Google Sheets as a repository with structured rubrics, Google Apps Script for orchestration with batch processing and JSON output validation, and Gmail for the automatic communication of HTML feedback. Operational performance was evaluated across six parallel courses (n=113 students) by measuring evaluation latency, system robustness, and feedback quality. The results show processing times ranging from 1:42 to 2:01 minutes per parallel course, with feedback communication at approximately 1 second per student, representing a 96.8% to 98.4% reduction in teacher time compared to manual methods. The system generates criteria-based feedback (a 1-4 scale per criterion) with specific justifications, actionable recommendations, and transparency through the inclusion of original answers. The solution proved to be scalable, replicable, and pedagogically aligned with formative assessment principles, redefining the teacher's role towards design and supervision functions while maintaining professional control over evaluative decisions.
Downloads
References
Ayanwale, M. A., Molefi, R. R., & Liapeng, S. (2024). Unlocking educational frontiers: Exploring higher educators’ adoption of Google Workspace technology tools for teaching and assessment in Lesotho dynamic landscape. Heliyon, 10(9), e30049. https://doi.org/10.1016/j.heliyon.2024.e30049
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (pp. 610–623). https://doi.org/10.1145/3442188.3445922
Black, P., & Wiliam, D. (1998). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80(2), 139–148
Bond, M., Khosravi, H., De Laat, M., Bergdahl, N., Negrea, V., Oxley, E., Pham, P., Chong, S. W., & Siemens, G. (2024). A meta systematic review of artificial intelligence in higher education: A call for increased ethics, collaboration, and rigour. International Journal of Educational Technology in Higher Education, 21, 4. https://doi.org/10.1186/s41239-023-00436-z
Brennan, R. L. (2003). Coefficients and indices in generalizability theory (CASMA Research Report No. 1). University of Iowa. https://education.uiowa.edu/sites/education.uiowa.edu/files/2021-11/casma-research-report-1.pdf
Carless, D., & Boud, D. (2018). The development of student feedback literacy: Enabling uptake of feedback. Assessment & Evaluation in Higher Education, 43(8), 1315–1325. https://doi.org/10.1080/02602938.2018.1463354
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487
Hift, R. J. (2014). Should essays and other “open-ended”-type questions retain a place in written summative assessment in clinical medicine? BMC Medical Education, 14, 249. https://doi.org/10.1186/s12909-014-0249-2
Huang, J., & Whipple, P. B. (2023). Rater variability and reliability of constructed response questions in New York state high-stakes tests of English language arts and mathematics: Implications for educational assessment policy. Humanities and Social Sciences Communications, 10, 860. https://doi.org/10.1057/s41599-023-02385-4
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y., Chen, D., Dai, W., Chan, H. S., Madotto, A., & Fung, P. (2023). Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12), Article 248. https://doi.org/10.1145/3571730
Jönsson, A. (2021). Analytic or holistic? A study about how to increase the agreement between teachers’ assessments. Assessment in Education: Principles, Policy & Practice, 28(4), 384–401. https://doi.org/10.1080/0969594X.2021.1884041
Jonsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review, 2(2), 130–144. https://doi.org/10.1016/j.edurev.2007.05.002
Lee, S. S., & Moore, R. L. (2024). Harnessing generative AI (GenAI) for automated feedback in higher education: A systematic review. Online Learning, 28(3), 82–104. https://doi.org/10.24059/olj.v28i3.4593
Moskal, B. M., & Leydens, J. A. (2000). Scoring rubric development: Validity and reliability. Practical Assessment, Research, and Evaluation, 7(1), 1–7. https://doi.org/10.7275/q7rm-gg74
Nicol, D. J., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218. https://doi.org/10.1080/03075070600572090
Pham, N. S. (2024). Using AppSheet and Apps Script to develop management and implementation applications for university education programs. VNU Journal of Science: Education Research, 40(1). https://doi.org/10.25073/2588-1159/vnuer.47362
Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189. https://doi.org/10.3102/0034654307313795
Tasdelen Teker, G. (2016). Using generalizability theory to investigate the reliability of performance assessments. International Journal of Assessment Tools in Education, 3(2), 166–178. https://files.eric.ed.gov/fulltext/ED585251.pdf
Williams, A. (2024). Delivering effective student feedback in higher education: An evaluation of the challenges and best practice. International Journal of Research in Education and Science, 10(2), 473–501. https://doi.org/10.46328/ijres.3404
Wisniewski, B., Zierer, K., & Hattie, J. (2020). The power of feedback revisited: A meta-analysis of educational feedback research. Frontiers in Psychology, 10, 3087. https://doi.org/10.3389/fpsyg.2019.03087
Xu, Q., Lin, L., & Wu, X. (2021). Implementing controlled digital lending with Google Drive and Apps Script: A case study at the NYU Shanghai Library. International Journal of Librarianship, 6(1), 37–54. https://doi.org/10.23974/ijol.2021.vol6.1.193
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Scientific Journal of Educational Innovation and Current Society "ALCON". ISSN 2960-8473
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
