Can Interpretation Predict Behavior on Unseen Data?
This work proposes and demonstrates an alternate objective for a new interpretability objective: understanding model internals to predict behavior and assess reliability under distribution shift, and successfully uses attention patterns to predict which rule each model follows on OOD data.