Vision Transformer Architecture for Simultaneous Detection and Severity Grading of Diabetic Retinopathy and Diabetic Macular Edema

Abstract

Diabetic retinopathy (DR) and diabetic macular edema (DME) are two major causes of preventable blindness. Despite their great success in DR and DME detection or grading individually, few methods have explored automated simultaneous grading of DR and DME from fundus photographs in clinic settings. In this work, we introduce DualHead-ViT, a multi-task framework with a pretrained ViT-B/16 backbone and a novel cross-task attention fusion module for jointly grading the severity of DR and DME. We project two separate CLS token vectors for DR and DME, respectively, and design a cross-task attention module to interact features of DR and DME bidirectionally. We validated our model using the Indian Diabetic Retinopathy Image Dataset (IDRID), employing stratified 5-fold cross- validation alongside CLAHE-preprocessed images and a class-weighted loss with ordinal regularization. Across 5 levels of DR severity, we report mean quadratic kappa of 0.802 ±0.010 and AUC of 0.831 ±0.018, and across 3 categories of DME risk, we report kappa of 0.759 ±0.067 and AUC of 0.895 ±0.026. Our findings support the feasibility of joint transformer-based grading for DR and DME using a single computational architecture.