Balanced Multimodal Federated Learning: An Efficient and Noise-Resilient Approach

Multimodal Federated Learning under Dual-Axis Modality Missingness

Multimodal federated learning (FL) supports collaborative modeling in privacy-sensitive health-sensing and medical settings, but realistic deployments often exhibit dual-axis modality missingness: clients have different modality sets, and individual samples may contain only subsets of the modalities available locally. Existing methods typically address these two axes separately. We propose Flux, a multimodal federated learning framework built around two complementary components. First, modality-aware confidence tempering learns sample-specific confidence for each modality through mask-aware unimodal supervision and fuses the confidence estimates from observed modalities into a sample-adaptive temperature that adjusts predictive sharpness according to evidence quality and completeness. Second, gradient-decoupled private adaptation applies this temperature only to a client-private prediction pathway, while training the shared federated model with a standard, untempered objective. This enables sample-specific, client-local confidence adaptation without allowing confidence-dependent gradients to perturb shared representation learning. Across four multimodal datasets, Flux achieves the highest average macro-F1 on every dataset, outperforming the strongest dataset-specific baseline by 0.8~2.2 points and by 1.6 points on average. Additional analyses demonstrate favorable calibration, temperature sensitivity to both modality missingness and input corruption, and more stable shared optimization under private-only tempering. Our code is available at https://github.com/AdibaOrz/Flux.

FedTaste: Topology-Aware Structural Transfer for Multimodal Federated Learning with Missing Modalities

FedTaste is proposed, a parameter-efficient framework for topology-aware structural transfer in Multimodal Federated Learning with missing modalities that avoids explicit modality imputation while preserving shared semantic structure across clients.

LAUA: Handling Missing Modalities and Unpaired Data in Multimodal Federated Learning

Multimodal federated learning enables multiple clients to collaboratively train models from distributed multimodal data while preserving data privacy. In realistic federated settings, multimodal samples are often missing or unpaired, and cross-modal heterogeneity across clients can hinder stable optimization. Many existing federated multimodal methods attempt to mitigate modality missingness by generating synthetic paired data through data augmentation or generative models. However, they typically rely on paired supervision or treat client updates uniformly, making them brittle under modality missingness and client-level variability. To address these challenges, we propose a federated multimodal learning framework (LAUA) for learning from a mixture of unimodal and multimodal clients. On clients, LAUA aligns representations in a shared variational latent space, where KL regularization yields a principled and lightweight confidence signal for estimating uncertainty. Unimodal clients learn transferable representations via self-supervised objectives, while multimodal clients additionally leverage task supervision and incorporate an internal distillation component to enhance cross-modal consistency and stabilize local optimization. On the server, LAUA performs uncertainty-weighted aggregation that adaptively down-weights unreliable client updates. Experiments on various datasets show that LAUA substantially mitigates performance degradation under modality missingness across retrieval and regression tasks, attaining up to 20% relative improvement in MRR for retrieval and up to 24.3% relative improvement in MSE for regression.

Robust Decentralized Federated Learning for Automatic Modulation Classification Under Impulsive Noise and Data Heterogeneity

Automatic modulation classification (AMC) is essential for enhancing the spectral efficiency and noncooperative communication capabilities of Internet of Things (IoT) systems. IoT devices are widely deployed in untrusted intelligent scenarios where data is edge-distributed and follows a nonindependent and identically distributed (non-IID) pattern, making applying traditional deep learning models directly challenging. Moreover, impulsive noise is prevalent in industrial and intelligent scenarios and dramatically degrades recognition accuracy. Federated learning (FL) has been extensively applied to privacy-preserving AMC tasks in recent years. However, the reliance of existing centralized FL architectures on a central server poses inherent security risks. To address these challenges, we propose a novel fully decentralized FL (DFL)-based AMC framework, termed DeKDAMC, which enables collaborative model training across edge devices without a central coordinator. Specifically, the framework incorporates a hybrid loss function with knowledge distillation (KD)-based temporal self-distillation to enhance local training consistency and alleviate optimization instability under heterogeneous data distributions. Furthermore, to enhance robustness, we embed bounded nonlinear function (BNF) modules within the network architecture to suppress the detrimental effects of impulsive noise. Extensive experiments demonstrate that the proposed DeKDAMC consistently achieves superior classification accuracy and enhanced stability, particularly under varying client counts and intermittent connectivity, validating its practical suitability for complex and security-sensitive IoT environments.

FedAMB: Adaptive Modality Balancing for Dominance-Robust Multimodal Federated Distillation

Mitigating Modality Quantity and Quality Imbalance in Multimodal Online Federated Learning

The Internet of Things (IoT) ecosystem produces massive volumes of multimodal data from diverse sources, including sensors, cameras, and microphones. With advances in edge intelligence, IoT devices have evolved from simple data acquisition units into computationally capable nodes, enabling localized processing of heterogeneous multimodal data. This evolution necessitates distributed learning paradigms that can efficiently handle such data. Furthermore, the continuous nature of data generation and the limited storage capacity of edge devices demand an online learning framework. Multimodal Online Federated Learning (MMO-FL) has been identified as a compelling strategy to address these requirements. However, MMO-FL faces new challenges due to the inherent instability of IoT devices, which often results in modality quantity and quality imbalance (QQI) during data collection. In this study, we systematically investigate the impact of QQI within the MMO-FL framework and present a comprehensive theoretical analysis quantifying how both types of imbalance degrade learning performance. To tackle these issues, we propose the Modality Quantity and Quality Rebalanced (QQR) algorithm, a prototype learning based method designed to operate in parallel with the training process. Extensive experiments on two real-world multimodal datasets show that the proposed QQR algorithm consistently outperforms benchmarks under modality imbalance conditions with promising learning performance.