Momentum is central to stochastic optimization in deep learning, including the training of large language models. Classical momentum methods typically rely on a fast exponential moving average (EMA) scheme, which discounts past gradients on a fixed timescale and may underweight older gradients that remain informative....
Hai-Feng Xu, Yi-Jin Ren, Chenyu Zhou et al.· 0 citations
Large language models increasingly automate optimization modeling from natural language problems, producing written mathematics, solver code and a returned report. Verifying one optimization problem across these artifacts requires semantic comparison: unequal models can attain the same objective, equivalent models can...
Rui Zhang, Chenyu Zhou, Jiang-Hao Lin et al.· 0 citations
Skill2Query is a framework that first parses a skill document into a Skill Knowledge Graph and then generates pseudo-queries through a three-stage process including style mimicking, query template generation, and parameter filling, which consistently improves sparse, dense, and skill-routing retrieval.
Lihui Ding, Zi-Han Guo, Bingwei Lu et al.· 2 citations
GraphBU is proposed, a graph-native generator whose basic unit is a local subproblem plus its interface whose basic unit is a local subproblem plus its interface that keeps graph statistics close to the source family, preserves feasibility on most datasets, and improves downstream Predict-and-Search training.
Xiaolei Guo, Chenyu Zhou, Jiang-Hao Lin et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
