Knowledge Tracing

The Knowledge Tracing Modules are designed to provide LASER Scholars with a comprehensive understanding and hands-on experience in various knowledge tracing methods used in digital learning platforms. Beginning with Bayesian Knowledge Tracing (BKT), scholars will build and explore classic BKT models using Python, gaining insights into its application across learning scenarios. The program then introduces Performance Factor Analysis (PFA) and Logistic Knowledge Tracing (LKT), where scholars will clean datasets and build LKT models, learning to analyze student performance involving multiple skills. Next, the modules cover Item Response Theory (IRT), equipping scholars with the principles and skills to validate educational assessments. Finally, the we wrap up this unit by diving into Deep Knowledge Tracing (DKT), where scholars will engage with deep neural network models, understanding their strengths and limitations. Throughout the modules, case studies, essential readings, and badge activities in ASSISTments will reinforce learning and application, preparing scholars to utilize these techniques effectively in educational research and practice.

	Github	Repository for Instructors
	Posit Cloud	Workspace for Learners

Module 1: Bayesian Knowledge Tracing

Bayesian Knowledge Tracing (BKT) (Corbett and Anderson 1994) is the most widely used student knowledge modeling framework within digital learning platforms. The BKT model provides decent-quality predictions of future performance within or outside the learning systems, interpretable models, meaningful parameters, and the ability to be applied to a range of learning situations. The goal of our Essential Readings and Case Study is to help LASER Scholars gain a theoretical understanding and practical experience with BKT. Our BKT Case Study is based on Zambrano, Zhang, and Baker (2024). You will use Python to build classic BKT models and explore some of the variations. Finally, you will complete the BKT Badge activity in ASSISTments and develop research questions utilizing a Large Language Model.

	Conceptual Overview	Bayesian Knowledge Tracing
	Code Along	BKT with ASSISTments BKT-BF walkthrough-PC BKT-BF walkthrough Mac
	Readings & Reflection	Essential Readings
	Case Study	Bayesian Knowledge Tracing with Python \| Answer Key
	Badge	Applying BKT in Practice
	Module Survey	Feedback Form After Finishing Module

Module 2: Performance Factor Analysis

Module 2 introduces Performance Factor Analysis (PFA) and logistic knowledge tracing (LKT) as alternative knowledge tracing methods. LKT utilizes Logistic Regression to investigate students’ performance. Unlike BKT, each item may involve multiple skills or knowledge components (KC). With the case study, you will learn to clean the dataset and build an LKT model. Our case study is based on the work of Tirronen and Tirronen (2020). This paper discusses the application of LKT in the programming education field and the case study will guide you to practice building your LKT model in R with the example dataset. Like module 1, you will complete the LKT Badge activity in ASSISTments and develop research questions utilizing a Large Language Model.

	Conceptual Overview	Logistic Knowledge Tracing and PFA
	Code Along	LKT walkthrough
	Readings & Reflection	Essential Readings
	Case Study	PFA case study
	Badge	LTK and PFA with ASSISTments
	Module Survey	Feedback Form After Finishing Module

Module 3: Item Response Theory

Module 3 wraps discuss Item Response Theory, a classic approach for assessment in tests. It is used to assess students’ current knowledge of a topic and it assumes no learning is occurring between items. Through exploring foundational principles, and building models in the case study, this module will equip you with valuable skills to understand the validity of educational assessments. Finally, the badge activity will help you reflect on how these techniques could be applied to research and practice.

	Conceptual Overview	Item Response Theory and ELO
	Code Along	Coming soon!
	Readings & Discussion	Essential Readings
	Case Study	IRT in R
	Badge	Apply IRT in practice
	Module Survey	Feedback Form After Finishing Module

Module 4: Deep Knowledge Tracing

Module 4 discusses the application of deep neural networks in knowledge tracing, called Deep Knowledge Tracing (DKT). It is a growing area and has dozens of variants. While deep neural networks are becoming popular and every paper claims good performance, we must be cautious and carefully understand this technique and its strengths and weaknesses before using it. Our essential readings and case studies cover selected current issues and approaches in Deep Knowledge Tracing (DKT). In the hands-on activities, you’ll be working to add a dataset to the implementation of the DKT model from Gervet et al. (2020). Finally, the badge activity will help you reflect on how these techniques could be applied to research and practice.

	Conceptual Overview	Intro to Deep Knowledge Tracing
	Code Along	Coming soon!
	Readings & Discussion	Essential Readings
	Case Study	DKT in Python
	Badge	Apply DKT in Practice
	Module Survey	Feedback Form After Finishing Module

Microcredential

The culminating activity for the Knowledge Tracing Modules is designed to provide you some space for independent analysis of a self-identified data source. To earn your KT Microcredential, you are required to demonstrate your ability to formulate a basic research question appropriate to a KT context, wrangle and analyze relational data, and communicate key findings. Your primary goal for this analysis is to create a simple data product that illustrates key findings by applying the knowledge and skills acquired from the essential readings and case studies.

Microcredential

Coming soon!

Essential Readings

Corbett, Albert T, and John R Anderson. 1994. “Knowledge Tracing: Modeling the Acquisition of Procedural Knowledge.” User Modeling and User-Adapted Interaction 4: 253–78.

Gervet, Theophile, Ken Koedinger, Jeff Schneider, Tom Mitchell, et al. 2020. “When Is Deep Learning the Best Approach to Knowledge Tracing?” Journal of Educational Data Mining 12 (3): 31–54.

Tirronen, Ville, and Maria Tirronen. 2020. “Estimating Programming Exercise Difficulty Using Performance Factors Analysis.” In 2020 IEEE Frontiers in Education Conference (FIE), 1–5. IEEE.

Zambrano, Andres Felipe, Jiayi Zhang, and Ryan S Baker. 2024. “Investigating Algorithmic Bias on Bayesian Knowledge Tracing and Carelessness Detectors.” In Proceedings of the 14th Learning Analytics and Knowledge Conference, 349–59.