Symbols and Neurons: A Review of Symbolic XAI in Deep Learning

Symbol Grounding in Neuro-Symbolic AI: A Gentle Introduction to Reasoning Shortcuts

This overview addresses this issue by providing a gentle introduction to RSs, discussing their causes and consequences in intuitive terms, and details methods for dealing with RSs, including mitigation and awareness strategies, and maps their benefits and limitations.

From Hybrid Mechanistic-Data-Driven Modeling Toward Neuro-Symbolic AI: What, Why, and How

This work bridges hybrid modeling and neuro-symbolic (NeSy) AI by reconstructing these designs as instances of NeSy interface and derives metrics: structural violation rate (SVR), measuring whether the learned belief respects the mechanistic structure; and belief dispersion (BD), measuring how concentrated the learned plausibility is, serving as a hybrid model's epistemic uncertainty in its mechanistic part.

Neurosymbolic Large Language Models: A Survey of Symbolic Integration, Reasoning and Explainability

LLMs have demonstrated strong language-learning and human-like response-generation capabilities, and they are increasingly used to support decision-making in high-risk sectors. However, their internal decision processes remain difficult to interpret, and their responses may lack transparency. The literature has explored numerous approaches to address transparency challenges in LLMs, including Neurosymbolic AI (NeSy AI). NeSy AI approaches were primarily developed for conventional neural networks and may not transfer directly to the distinctive characteristics of LLMs. Consequently, there is a limited systematic understanding of how symbolic AI can be effectively integrated into LLMs. This paper aims to address this gap by first reviewing established NeSy AI methods and then proposing a novel taxonomy of symbolic integration in LLMs, along with a roadmap to merge symbolic techniques with LLMs. The taxonomy organises the literature across four dimensions: (1) the stage of LLM development at which symbolic information is integrated; (2) the coupling mechanism; (3) the architectural paradigm; and (4) the algorithm-level or application-level perspective. The review identifies commonly used benchmarks, recent advances and important research gaps, and uses these findings to outline directions for future research. By highlighting the latest developments and notable gaps in the literature, it offers practical insights for implementing frameworks for symbolic integration into LLMs to enhance transparency.

Mathematical Principles and Experimental Discoveries of the Emergence of Symbolic Patterns in Artificial Neural Networks

It is shown that across a broad class of ANNs trained on diverse tasks, their inference logic can indeed be reformulated as sparse symbolic interactions, and two common mathematical criteria lead to the emergence of such sparse symbolic interactions.

An Overview of Neuro-Symbolic Frameworks for Software Development

Interpretability in the Era of Large Language Models: Mechanistic Methodology, Empirical Practices, and Applications

The rapid evolution of Large Language Models (LLMs) has brought unprecedented capabilities across reasoning, coding, and multimodal tasks. However, as performance scales, their opaque ''black-box'' nature raises a critical challenge: How can we trace the origins of emergent intelligence, and more importantly, how can we leverage these internal mechanisms to guide model optimization? This tutorial provides a comprehensive, end-to-end view of LLM interpretability, transitioning from microscopic neural analysis to macroscopic application and deployment. It is systematically organized into five core sections: i) Unlocking the Black Box: We begin with the evolution of LLM interpretability and highlight recent breakthroughs from leading research teams. ii) Methodology: We present a rigorous overview of foundational theories (e.g., mathematical framework for transformer, biological mechanisms in LLMs) and essential methods (e.g., path patching, logit lens, and neuron description). iii) Anatomy of LLMs: Using advanced techniques to decode internal semantic features, neural circuits, and complex behaviors, we interpret how models perform reasoning, factual recall, and in-context learning. iv) Applications: We show how to transfer interpretability insights into actionable improvements across the LLM pipeline, including interpretability-guided data synthesis (data value scoring, corpus filtering, and activation-based data diagnosis). We also present Pinpoint Training and Steering for precise capability gains, and Pinpoint Quantization for extreme low-bit compression with minimal capability loss. v) Advanced Topics: We conclude by exploring how these interpretability paradigms scale and inspire the design of frontier architectures, agentic systems, and thinking models. In this tutorial, researchers and engineers will gain the theoretical frameworks and practical engineering toolkits needed to understand, steer, and efficiently deploy LLMs in real-world production environments.