O. Khobor

GENERALIZING TINYBERT WITH CROSS-ATTENTION, LORA AND BI-GRU FOR DEEP LEARNING OVER NON-NUMERIC DATA: BEYOND FAKE NEWS DETECTION

A universal hybrid architecture integrating the compact TinyBERT transformer, a cross-attention mechanism, low-rank adaptation (LoRA), and a bidirectional gated recurrent unit (Bi-GRU) is proposed, capable of delivering high accuracy, interpretability, and computational efficiency for heterogeneous textual data in mult...