Knowledge tracing algorithms are embedded in Intelligent Tutoring Systems (ITS) to keep track of what students know and do not know, in order to better focus practice. Due to costly in-classroom experiments, we explore the idea of using machine learning models that can simulate students’ learning process. We conduct experiments using such agents generated by Apprentice Learner (AL) Architecture to investigate the online use of different knowledge tracing models (Bayesian Knowledge Tracing and the Streak model). We were able to successfully A/B test these different approaches using simulated students. An analysis of our experimental results revealed an error in the implementation of one of our knowledge tracing models that was not identified in our previous work, suggesting AL agents provide a practical means of evaluating knowledge tracing models prior to more costly classroom deployments. Additionally, our analysis found that there is a positive correlation between the model parameters achieved from human data and the param- eters obtained from simulated learners. This finding suggests that it might be possible to initialize the parameters for knowledge tracing models using simulated data when no human student data is yet available.

Relevant Publications

Zhang, Q., Chen, Z., Lalwani, N., MacLellan, C.J. (2022). Modifying Deep Knowledge Tracing for Multi-step Problems. In Proceedings of the 15th International Conference on Educational Data Mining.

Zhang, Q., MacLellan, C.J. (2021). Going Online: A simulated student approach for evaluating knowledge tracing in the context of mastery learning. Proceedings of the Fourteenth International Conference on Educational Data Mining.

Zhang, Q., MacLellan C.J. (2021). Investigating Knowledge Tracing Models using Simulated Students. AAAI2021 Spring Symposium on Artificial Intelligence for K-12 Education.