FETRA: A Federated Transformer with Dynamic Attention Model for Energy Forecasting in IoBT

Abstract

Recently, the integration of Internet of Things (IoT) into buildings has sparked a technological revolution known as the Internet of Building Things (IoBT). Such new paradigm connects devices, sensors, and systems for intelligent control. Particularly, IoBT has advanced energy forecasting and enabled real-time data analysis and predictive optimization for smarter and more efficient energy management. This paper proposes FETRA, a FEderated TRansformer with Attention-based client weighting and adaptive FedProx regularization for smart building energy prediction. FETRA addresses data heterogeneity through dynamic client importance assignment, captures long-term temporal dependencies via specialized attention mechanisms, and enhances convergence stability under non-IID distributions. Experimental evaluation on the ASHRAE dataset with 100 heterogeneous buildings demonstrates that the Informer model achieves superior performance with 94.18% prediction accuracy, MSE of 892.45, RMSE of 29.87, and $\mathrm{R}^{2}$ of 0.94, outperforming baseline techniques. In addition, the edge-fog-cloud architecture adapted in FETRA enables privacy-preserving distributed energy forecasting while maintaining computational efficiency. Results confirm FETRA's effectiveness in balancing prediction accuracy, system robustness, and scalability for real-world deployment in decentralized building energy management systems.