CSA-Net: A Type-Aware Dual-Tower Neural Model for Imbalanced Tabular Credit Default Prediction

Abstract

Credit default prediction under class imbalance requires attention to both ranking quality and fixed-threshold behavior. This study evaluates imbalanced tabular credit default prediction on the Default of Credit Card Clients (DCCC, approximately $4{:}1$ ) and Give Me Some Credit (GMSC, approximately $14{:}1$ ) benchmarks under a shared leakage-controlled protocol. Supervised preprocessing rules, including supervised binning and Weight of Evidence (WOE) encoding, are fitted on the training partition only and then applied unchanged to validation and test data. We compare WOE-based logistic regression, XGBoost, LightGBM, the Self-Attention and Cross-Network (SACN) baseline, and the proposed Context-State Attention Network (CSA-Net). CSA-Net is a type-aware dual-tower architecture that routes discrete bin identifiers to a Transformer-based Context Tower and WOE-scaled inputs to a Deep & Cross Network v2 (DCN-V2)-style State Tower, with a cross-attention bridge as a structured interaction module. The results do not show uniform superiority over all baselines. Tree-based models remain the strongest ranking baselines, and WOE-based logistic regression remains a competitive scorecard-style anchor. The clearest neural separation appears under Focal Loss at the fixed threshold of 0.5. Under binary cross-entropy and Weighted BCE, CSA-Net and SACN are close, whereas under Focal Loss SACN shows near-collapse in fixed-threshold default detection on GMSC. CSA-Net maintains high recall in this setting, while NoCross, its bridge-removed ablation, closely tracks the full model. These findings indicate that the loss function mainly changes the fixed-threshold operating profile.