Fine-tuning large language models at scale is increasingly limited by the memory cost of backpropagation. Zeroth-Order (ZO) optimizers reduce this burden by relying only on forward function evaluations, but they still exhibit a substantial performance gap to first-order (FO) optimizers such as Adam. We argue that this...
Understanding the dynamics of large quantum systems is hindered by the curse of dimensionality. Statistical learning offers new possibilities in this regime through neural network protocols and classical shadows, while both methods have limitations: the former suffers from incompatible dataset construction rules, resul...
Yuxuan Du, Yi-Bo Yang, Tongliang Liu et al.· IEEE Transactions on Pattern...· 10 citations
Extensive experiments across image classification, natural language understanding, and natural language generation tasks show that AdaMSS achieves comparable performance to full fine-tuning and outperforms other parameter-efficient fine-tuning methods in most cases, all while requiring fewer trainable parameters.
Jingjing Zheng, Wang-Long Lu, Yiming Dong et al.· Neural Information Processin...· 0 citations
This work proves the first near-optimal lower bound for arbitrary adaptive randomized algorithms throughout both accuracy regimes of exact value Lipschitz convex optimization, and develops a posterior mean energy method for adaptive exact max observations.
Haihan Zhang, Chen-Heng Zhang, Zhiquan Qi et al.· 1 citation
This work revisits client drift from a novel frequency-domain perspective and uncovers a critical Spectral Bias of Drift: inter-client gradient divergence is predominantly concentrated in low-frequency components which encode client-specific distributional shifts, while high-frequency components representing fine-grain...
Liyang Yuan, Yibo Yang, Dandan Guo et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.