OCR Table Extraction and Compliance Verification of Legal Documents Based on Multimodal Transformer

Abstract

Accurate extraction and intelligent interpretation of structured information from complex documents remain challenging due to heterogeneous data modalities, intricate layouts, and long-range semantic dependencies. This study proposes a multimodal document understanding framework integrating optical character recognition (OCR), multimodal Transformer encoding, graph neural networks (GNNs), and reinforcement-learning-based reasoning. A dedicated OCR and layout-analysis module is first employed to acquire textual, visual, and spatial information from complex document images. Subsequently, a multimodal Transformer encoder performs joint representation learning by fusing visual features, semantic content, and layout embeddings to achieve robust table structure reconstruction and structured information extraction. To model semantic relationships among extracted entities, a relational graph convolutional network is constructed for heterogeneous information representation and propagation. Furthermore, a rule-guided reinforcement learning mechanism is introduced to optimize reasoning paths and improve decision accuracy in complex verification tasks. Experimental results demonstrate that the proposed framework achieves a GriTS score of 0.927 for table reconstruction and an F1 score of 93.1% for end-to-end verification, outperforming existing document-understanding approaches while significantly improving processing efficiency. The proposed framework provides an effective methodology for multimodal information fusion, graph-based information propagation, intelligent signal representation, and adaptive reasoning in complex information-processing systems, offering potential applications in intelligent sensing, communication-oriented information analysis, and distributed decision-support environments.