Apprentice tutors is a platform for authoring and deploying tutors capable of providing personalized education of AI concepts at scale. We engineered the Apprentice tutors to provide adaptive problem capabilities using Bayesian Knowledge Tracing (BKT) to engage students with question types targeting knowledge components needing the most improvement. We aim to provide this tutor as a supplement to in-class instruction. Our work showcases the promise of expanding equitable and personalized AI education throughout higher education to enable students to learn in scenarios where a human-AI expert instructor might not be readily available.

tutor interface

Recomendations in Building Intelligent Tutors

To uncover the core themes from our focus group data, we employed a thematic analysis approach as outlined by Braun and Clarke (2006) Braun & Clarke, 2006, to capture the nuanced perspectives of both students and teachers regarding the usability, functionality, and overall impact of the Apprentice Tutors platform. This process informed our design recommendations and strategic improvements stated below.

Theme Key Intelligent Tutor Themes Participant Quotes from Focus Groups
AI support Feelings Users are concerned that tutors will decrease the overall amount of “human” and “human-like” interaction, which they think will decrease learning. S: I think one thing that’s true of AI, whether that’s a tutor or not, is it takes out the human factor. The AI isn’t gonna understand that there might be some barriers from it trying to walk you through something that it can’t see.
AI support Feelings Users like the “superhuman” aspects of tutors, where they provide support a human teacher cannot. T: The positive aspect for me is knowing that there’s something out there to help strengthen those concepts that students have seen before but haven’t seen in a while and they may need some more guided practice on them if they’re struggling.
Tutor Hints Support Users find current hints and feedback to be insufficient and want clearer, more detailed explanations that help students understand what they did wrong and how the tutor reached its answer. S: So I click next hint and it just gives me the answer and I don’t know how it got that answer so that it didn’t click for me. I was just, I kind of gave up on using it.
Tutor Hints Support Users want tutors to explicitly connect and link to relevant concepts and explanatory content. T: When I finally went into the OpenStax textbook and I’m like, OK, that’s what they’re referring to as B and C and I’m like, they’re defined as this and this—I think that would be helpful if that were clearly displayed.
Tutor Hints Support Users would like tutors to link to explanatory videos to support their learning. S: YouTube as well [for getting unstuck on a problem]. There’s a lot of teachers on there and sometimes they just explain it differently, and I get it from watching several different peoples’ methods.
Tutor Hints Support Users think it would be helpful if they could ask the tutor questions and have dialogue-based interactions. S: [What would be a characteristic of an ideal tutor?] Something that’s gonna have a conversation with you rather than feeling like I’m just plugging something into a calculator.
Tool Adoption Teachers want more tutor content and the ability to create and customize it. T: I don’t know if I have the skills to build my own tutor, but it would be nice to create my own tutor problems.
Tool Adoption Teacher adoption depends on understanding how tutors work and seeing alignment between tutors and course content. T: If I am confused on what the tutor interface says, how can I tell the students to use the tutors?
Tool Adoption Users are willing to provide time and data to improve the tutor. T: Two teachers – when asked if they’d collaborate with the development team to address tutor deficiencies: “Yes, absolutely.”
Usability & Value Considerations Users are more likely to adopt tutors if they see a clear benefit/incentive and if they are “reminded.” T: I don’t think I have done a good job reminding students to do the tutor.
Usability & Value Considerations Users find tutors frustrating & confusing; usability must be improved and more support (e.g., tutorial videos) provided before their full value is realized. T: Now the concept is great… I don’t delve deeply into simplifying radicals because I don’t have the time. But if it was one that was easy to understand and get through—and user friendly—Amen.
Usability & Value Considerations Users had many usability/bug issues that produced confusion. T: Let’s pull up the exponents product rule; there was no way to show what we need to type in the first box. In this particular problem, what is the correct answer? (The hint box showed the answer in LaTeX notation, which was hard to understand.)
Usability & Value Considerations Teachers are frustrated they cannot see who is using the tutor, how tutor use relates to student progress/learning, or evidence of tutor effectiveness. T: I was also frustrated—I couldn’t see who was accessing it. I would have to rely on students [to tell me].
Usability & Value Considerations Teachers found simpler tutors less confusing. T: The tutor is pretty; it kind of speaks for itself since it’s pretty simplistic.
Features Users had several features they liked and also suggested new possible features. T: Is there a possibility that if you get the question incorrect after so many times, it tells you how to move on?
Features There are key subpopulations (minors and neurodiverse users) that should be identified and designed for. T: I have a learning disability called Dyscalculia. And the unfortunate bit is the AI tutor doesn’t seem to take that into consideration.

Relevant Publications

Gupta, A., MacLellan, C.J. (2021). Designing Teachable Systems for Intelligent Tutor Authoring. AAAI2021 Spring Symposium on Artificial Intelligence for K-12 Education.

MacLellan, C.J., Koedinger, K.R. Domain-General Tutor Authoring with Apprentice Learner Models. Int J Artif Intell Educ 32, 76–117 (2022). https://doi.org/10.1007/s40593-020-00214-2.

Gupta, A., Siddiqui, M., Smith, G., Reddig, J., MacLellan, C. Intelligent Tutors for Adult Learners: An Analysis of Needs and Challenges. arXiv preprint arXiv:2412.04477 (2024).

Gupta, A., MacLellan, C. Intelligent Tutors Beyond K-12: An Observational Study of Adult Learner Engagement and Academic Impact. arXiv preprint arXiv:2502.16613 (2025).