Memory-Efficient Probabilistic Neuro-Symbolic Integration for Explainable Natural Language Inference Using Transformer-Based Foundation Models
TL;DR
A memory-optimized probabilistic neuro-symbolic hybrid architecture that unifies transformer-based neural networks with logic-based symbolic reasoning systems within memory-constrained systems indicates that neuro-symbolic systems within memory-constrained systems can achieve both explanation needs and foundation models’ performance requirements.
Abstract
Background: Transformer-based foundation models have achieved state-of-the-art results in various natural language inference benchmarks, but their decision-making processes remain largely unexplainable. Addressing the ’explainability gap’ is crucial for responsible AI adoption in highrisk industries that require transparency and trustworthiness. Furthermore, the combination of neural pattern matching with structured symbolic reasoning in resource-constrained scenarios is an important open problem. Objective: This study aims to present a memory-optimized probabilistic neuro-symbolic hybrid architecture that unifies transformer-based neural networks with logic-based symbolic reasoning systems. Methods: We use the e-SNLI dataset that provides human-written natural language explanations and reasoning highlights as training targets, and finetune the BERT transformer-based language model with an emphasis on gradient checkpointing, mixed-precision (FP16) training, and layer freezing for optimal resource utilization/reasoning tradeoffs. All experiments were performed on an NVIDIA GPU with 8–12 GB VRAM and CUDA-compatible hardware. Results: The proposed framework achieves 80.6% accuracy on 3-way NLI classification (contradiction, entailment, and neutral) with 0.806 precision, recall, and F1 scores on each class, and detailed class-level analysis shows high performance on entailment recognition (F1 = 0.912) and contradiction detection (F1 = 0.902), but slightly lower performance on neutral cases (F1 = 0.864). Ablation studies and confidence distributions of the model predictions indicate that memory-optimized models can maintain competitive performance and be deployed on resource-constrained devices, reducing GPU memory usage by ~60%. Conclusions: The results indicate that neuro-symbolic systems within memory-constrained systems can achieve both explanation needs and foundation models’ performance requirements, representing an important step in creating more trustworthy AI for NLP.