A deep learning model for detection and staging of osteonecrosis of the femoral head on x-ray images: a retrospective, large-scale, multicentre study in China.
TL;DR
A deep learning-based diagnostic model for the automatic detection and staging of osteonecrosis of the femoral head using x-ray images is developed and its performance with clinicians is compared to allow them to improve their diagnostic performance.
Abstract
Background
Accurate detection and staging of osteonecrosis of the femoral head (ONFH) are essential for effective treatment yet frequently require specialist expertise and advanced equipment. To address this requirement, we aimed to develop a deep learning-based diagnostic model for the automatic detection and staging of ONFH using x-ray images, and compared its performance with clinicians to allow them to improve their diagnostic performance.
Methods
In this retrospective, large-scale, multicentre study, we developed our deep learning model, Osteonecrosis Artificial Intelligence Diagnostician (OsteoAID), which was based on the You Only Look Once version 8 (YOLOv8) detection network and trained on a multicentre x-ray dataset. The dataset, dated from Jan 1, 2015, to Dec 31, 2023, and obtained from 16 399 patients across six Chinese tertiary hospitals, included 20 722 anteroposterior hip x-ray images of 28 785 hips, including hips with ONFH, healthy hips, and hips with other conditions radiographically similar to ONFH. To establish a rigorous ground truth for model training and evaluation, expert orthopaedists annotated each image to differentiate ONFH from confounding diseases and additionally classified the confirmed ONFH lesions on the basis of the Ficat and Arlet staging system. To verify the diagnostic performance of OsteoAID, we compared the model's accuracy, sensitivity, specificity, F1 score, and area under the curve (AUC) with those of the original YOLOv8 model on the internal testing set for both the staging and detection tasks. Furthermore, the performance of OsteoAID was compared with the performance of radiologists and orthopaedists on two external testing sets, and its diagnostic performance was also benchmarked against MRI-the gold standard for ONFH diagnosis-with a third external testing set in which x-ray images were staged by corresponding MRI scans. Diagnostic metrics were compared with those of clinicians using t tests and McNemar's tests, with significance set at p<0·05.
Findings
On the internal testing set, OsteoAID improved the staging accuracy from 87·42% (95% CI 85·88-89·20) to 96·44% (94·73-97·17) and the AUC from 88·54% (86·74-90·37) to 96·94% (95·75-97·98) compared with the original YOLOv8 model. For detection the model had a mean average precision of 92·97% (91·73-94·43) at an intersection-over-union threshold of 50%. In comparisons with clinicians on the external testing set I, the model showed staging accuracy of 96·60% (94·27-97·81), which was significantly higher than radiologists (74·96% [73·38-76·85]; p<0·0001) and junior orthopaedists (83·98% [81·27-84·06]; p<0·0001) and similar to that of expert orthopaedists (96·54% [94·18-97·79]; p=0·20). Furthermore, model-assisted diagnostics improved the accuracy of junior orthopaedists by an absolute difference of 12·70% and of radiologists by 21·47%. Although the accuracy improvement for expert orthopaedists was non-significant (an absolute difference of 0·74%; p=0·080), the diagnostic time was substantially reduced across all groups. On external testing set II (which included challenging cases: low-quality x-ray images affected by improper exposure, poor contrast, incomplete field of view, or foreign-object shadows, as well as smartphone photographs of x-ray films), the model showed robust performance even under extreme conditions, achieving an overall staging accuracy of 93·76% (93·09-95·82). Additionally, combining x-ray imaging with the deep learning model produced diagnostic results similar to MRI on external testing set III with an overall accuracy of 96·63% (95·68-97·86) and an AUC of 97·33% (96·65-98·75).
Interpretation
The developed deep learning model shows expert-level diagnostic performance in the dual tasks of detecting and staging ONFH by use of anteroposterior x-rays, which can substantially enhance both the diagnostic proficiency and efficiency of clinicians. Consequently, the model could serve as a highly accessible decision-support tool to optimise clinical triage and guide appropriate treatment strategies.
Funding
National Natural Science Foundation of China, National High Level Hospital Clinical Research Funding, and Beijing Natural Science Foundation.