Adversarially-Robust and Explainable Reinforcement Learning for Fraud Detection in Digital Banking
Abstract
Reinforcement learning (RL) has recently emerged as a viable adaptive replacement for traditional rule-based and supervised methods for detecting fraud perpetrated against banks that operate online. Nevertheless, RL-based fraud detection systems have always been evaluated in static, non-aggressive environments; hence, we cannot tell the true effectiveness of these systems. To remedy the situation, we propose a self-play system based on adversarial RL, which consists of a defender agent trained to identify transactions as either legitimate or fraudulent and an attacking agent trained to change fraudulent transactions in such a way that they will not be identified as fraudulent. To demonstrate the system’s capabilities, we used the dataset from the PaySim mobile-money simulator and demonstrated that even though the defender agent trained in a standard way achieved quite good results in identifying transactions as legitimate or fraudulent (92.8% precision, 89.7% recall, F1 = 0.912), it does not allow to avoid being successfully attacked in 51.3% percent of cases. Collecting adversarial examples over several rounds eliminates this problem and decreases the evasion rate to 22.4%, while the precision on clean data drops to 72.6%. This situation shows that there is a trade-off phenomenon between robustness and accuracy. We also indicate that a traditional gradient boosting classifier produces nearly ideal metrics for clean data, yet is attacked more effectively (71.6%) than both RL methods, revealing the misleading nature of the said clean data accuracy, which is not a good indicator for evaluating adversarial robustness in this case. Finally, we introduce SHAP-based explainability for the last defender to get per-transaction and overall feature attribution and overcome the unaccountability issue of the previous RL fraud detection systems.