Recent developments have demonstrated the efficacy of enhancing the reliability and reasoning ability of Large Language Models (LLMs) through the integration of Knowledge Graphs (KGs). However, previous works have not fully considered the challenges present in expert domains, including the prevalence of professional te...
Shuo-Ling Liu, Hui Wu, Kun Yi et al.· IEEE Transactions on Knowled...· 0 citations
Since the advent of knowledge distillation, KL divergence has been the standard loss in distillation. Recently, on-policy distillation (OPD) has emerged as an efficient post-training paradigm for LLMs. As a distillation method, OPD naturally inherits KL divergence as its standard loss. However, in this work, we find th...
Wen-Ze Lin, Ji-Yuan Long, Jia-Le Zhao et al.· 0 citations
SAGE (Stability-Aware Gradient Efficiency), which maintains difficulty-stratified candidate pools refreshed during training and selects pairs within each pool by a forward-pass signal-to-curvature score, outperforms full-data and size-matched baselines while producing substantially smoother optimization trajectories.
Hui Wu, Hengyi Cai, Jin-Man Zhao et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.