Beyond the Black Box: Neuro-Symbolic Integration for Interpretable Video-Based Reinforcement Learning

Abstract

: This paper introduces a cognitively inspired neuro-symbolic framework for extracting interpretable world models from raw video streams in dynamic 2D environments. While traditional end-to-end deep reinforcement learning systems often function as opaque “black boxes,” our approach decouples visual perception from policy learning to enhance transparency. By leveraging Core Knowledge theory (specifically object persistence, physical causality, and agent representation), the system transforms visual patches into structured symbolic entities and governing interaction rules. The core of this architecture is a symbolic module that reconstructs persistent objects, infers parametric motion laws (such as velocity inversion upon contact), and incrementally consolidates class-specific behaviors into a compact knowledge base. This world model instantiates a domain-agnostic environment wrapper, enabling a Deep Q-Network to operate on symbolic state vectors rather than pixels. We further propose a domain-agnostic agent that develops complex behaviors driven by a composite intrinsic reward based on causal impact and event-driven curiosity. Experimental evaluations on Arkanoid and Pong demonstrate that this framework achieves near-optimal performance without task-specific external rewards. On Arkanoid, the agent matches the win rate of fully supervised models while remaining robust to structural modifications, such as changes in ball size or brick configuration. In Pong, the same mechanism transfers without architectural adjustments, consistently improving survival times. These results provide a transparent, generalizable alternative to dominant AI paradigms with good performance across varied environmental conditions.