Explainable Transformer Models for Insider Threat Detection in Enterprise Systems

Abstract

Insider threats are difficult to detect because malicious activity is embedded in otherwise legitimate user behavior, and analysts need interpretable justifications as well as accurate alerts. This paper proposes XFT, an explainable transformer framework operating on tokenized behavioral evidence from heterogeneous enterprise logs covering authentication, removable device, file, email, and web proxy events. Each user-day is a sequence of feature tokens combining a learned value embedding with a feature identity embedding, and a transformer encoder with a classification token aggregates this evidence through multi-head self-attention. A hybrid variant fuses the transformer score with a gradient boosting score using a validation-selected weight, and an explanation layer reports attention, gradient-times-input saliency, and SHAP attributions with an agreement check. On 693,649 user-day records from CERT r5.2 the hybrid model reaches an Area Under the Receiver Operating Characteristic Curve (ROC AUC) of 0.9914, detects 92.7% of malicious user-days at a 1% false-positive budget, and confirms 158 of 261 malicious test days at a false-positive rate of 0.003%, with a paired stratified bootstrap over 300 replicates confirming the gain over gradient boosting alone. On the CERT r4.2 user-level benchmark, the strongest configurations reach 97.7% accuracy with a ROC AUC up to 0.9861. Attention correlates moderately with gradient saliency but weakly with SHAP, and a deletion audit shows that removing the three features ranked highest by attention reduces the flag score by 41%, compared with 12% for random removal.