Bayesian Policy Optimization over Conceptual Lattices for Interpretable Personalized Learning

Abstract

Objectives: To develop a framework that integrates pedagogical structure with uncertainty-aware decision-making for personalized learning in smart educational environments, addressing the limitations of current deep reinforcement learning approaches that treat curricula as unstructured sequences. Method: The study formalizes the learning domain as a concept lattice—an order-theoretic structure derived from formal concept analysis that encodes prerequisite relationships. Within this structured state space, a Bayesian reinforcement learning agent using Thompson sampling maintains joint posterior distributions over the learner's latent knowledge state and the uncertain reward associated with each instructional action. The framework was evaluated on the ASSISTments 2012-2013 dataset (4,317 problems, 112 knowledge components, 334,416 interactions) and Eedi (98 concepts, 7,547 interactions)—and validated against four baseline methods: Standard Thompson Sampling, Graph-Constrained RL, Bayesian RL, and Static Policy. Findings: The proposed Structured Thompson Sampling (STS) framework achieved a 15.2% improvement in average skill gain over standard Thompson sampling on ASSISTments and a 14.8% improvement on Eedi, demonstrating consistent performance across datasets. The system demonstrated faster convergence with approximately 32% fewer training interactions. The system outputs well-calibrated uncertainty estimates with an expected calibration error of 0.036, supporting interpretable decision-making for educators. The Pedagogical Coherence Score of 0.96 confirms that STS respects prerequisite relationships, while ablation studies revealed that both the lattice structure and Bayesian optimization contribute significantly to performance. Novelty: This work presents the first integration of formal concept analysis with Bayesian reinforcement learning for pedagogical sequencing, providing a mathematically rigorous foundation for personalized learning that combines structural validity with quantifiable confidence estimates. The framework bridges the critical gap between pedagogical coherence and uncertainty-aware decision-making in adaptive educational systems. Keywords: Bayesian Reinforcement Learning, Personalized Learning, Concept Lattice, Thompson Sampling, Pedagogical Sequencing, Uncertainty Quantification, Smart Learning Environments, Adaptive Educational Systems