Explainable AI studies for object detection using transformer-based vision models

Abstract

Historically, convolutional methods such as YOLO, SSD, and R-CNN were the driving force behind object detection and localization research. In recent years, however, transformer-based approaches like DETR have fundamentally shifted the paradigm for model development. Understanding how objectness is learned can inform how we train these models more effectively. The attention modules used within transformer models provide a useful method for generating saliency maps, thereby exposing which parts of an image the model learns to focus on. However, very large transformer models e.g., with over 30 complex components, make determining meaningful feature contribution difficult. This work explores selecting encoded image features from different transformer blocks and passes them to an object detection head. This technique allows us to determine which transformer blocks contribute most to object recognition. We then visualize the attention weights of the highest-contributing blocks to create saliency maps that specify which aspects of the input image receive the most focus in each attention block. This allows researchers to better understand why objects are detected in certain locations and not others, which is particularly useful for geospatial applications. We show that careful selection of feature extraction locations in a model’s architecture can improve object detection performance by 0.323 in terms of the mAP 50 score. This means that we can better leverage pretrained models without modifying their architecture.