Truncated SVD-BLS: Enforcing Flat Minima for Robust Broad Learning via Relative Flatness

Abstract

Broad Learning System (BLS) achieves efficient training through random feature mapping and closed-form pseudoinverse solutions. However, the standard BLS suffers from limited generalization performance owing to its high sensitivity to feature perturbations, particularly when the feature matrix contains near-zero singular values. Motivated by the Relative Flatness theory, which establishes that feature robustness directly governs generalization bounds, this study proposes the Truncated SVD-BLS, a method that enforces flat minima by discarding small singular value directions. We rigorously prove that the proposed method reduces the Lipschitz constant of the loss function with respect to feature perturbations from order of the reciprocal of the smallest singular value to order of the reciprocal of the retained singular value, thereby tightening the generalization error bound. Experimental results on five benchmark datasets demonstrate that Truncated SVD-BLS achieves an accuracy comparable to or higher than that of standard BLS while exhibiting significantly improved robustness against feature noise.