Privacy Preserving using Decentralized Federated Learning with Partial Dense-layer Weight Sharing
TL;DR
Empirical findings indicate that DFL-PHS, in many configurations, competes with or outperforms CFL, attaining F1-scores of 0.965 and recall rates surpassing 0.98, even under minimal weight-sharing conditions.
Abstract
Federated Learning (FL) enables participants to collaboratively train a model without sharing raw data. Traditional FL models depend on a central server and full model sharing, leading to a significant issue in healthcare related to privacy concerns and communication overhead. However, to address such issues, we introduce Decentralized Federated Learning with Partial Head Sharing DFL-PHS, a framework that eliminates the central server, allowing peer-to-peer model updates sharing. Instead of sharing entire model parameters, participants only share a subset of weights from the final dense layers, minimizing information breach while maintaining the model’s performance. We apply our proposed framework to COVID-19 chest X-ray binary classification, where we conduct a comparative study across four scenarios: local training, centralized FL (CFL), single-site training (SST) and DFL-PHS. Evaluations are conducted on three dataset sizes (small, medium, large) and four partial sharing ratios (25%, 50%, 75%, 100%). We also assess privacy through a Membership-Inference Attack (MIA), with only 25% head sharing, the attack operates at or below chance (ROC-AUC 0.37 with advantage -0.30 on the small scale and ROC–AUC ≈0.50\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\approx 0.50$$\end{document} with advantage ≈0.00\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\approx 0.00$$\end{document} on the large scale), indicating no reliable membership signal. Empirical findings indicate that DFL-PHS, in many configurations, competes with or outperforms CFL, attaining F1-scores of 0.965 and recall rates surpassing 0.98, even under minimal weight-sharing conditions. These results validate DFL-PHS as a privacy-preserving and scalable system for distributed FL. While evaluated on a single radiography dataset, our framework is dataset-agnostic; we plan broader validation on non-medical and other benchmarks as future work.