In this article, we study deep nonparametric regression trained via stochastic gradient Langevin dynamics (SGLD), a variant of stochastic gradient descent that injects carefully calibrated Gaussian noise during optimization. Using a PAC-Bayesian analysis tailored to deep ReLU networks, we establish an excess-risk bound...
Supervised fine-tuning (SFT) is a key component of language model post-training, enabling effective task adaptation and serving as an important foundation for subsequent reinforcement learning. Recent methods seek to improve SFT by introducing token-level weights into the training objective, where these weights are typ...
Representation-based Masked Diffusion Model (RMDM) is proposed, a framework that leverages the text representation to explicitly encode global semantics and help to parallel update tokens more precisely, and significantly improves generation quality, particularly in aggressive few-step sampling regimes.
Yang Hu, Ding Huang, Xue-Yu Zhou et al.· 0 citations
Pairwise Likelihood MIA (PL-MIA), a unified method that combines a Gaussian likelihood-ratio (GLR) statistic with population calibration and the Cauchy combination test, is proposed, demonstrating how statistical principles can turn noisy model outputs into more powerful, calibrated, and reproducible evidence for membe...
Sheng-Jie Niu, Ze-Bin Yun, Ye-Heng Ge et al.· 0 citations
Word-level Probability MIA (WPMIA) is proposed, a statistically principled MIA for strict black-box privacy auditing that estimates word-level generation probabilities via Monte Carlo sampling with local kernel smoothing, then aggregates these estimates into a sequence-level likelihood estimator.
LLM-based coding agents have significantly advanced automated software issue resolution, yet they remain highly prone to factual errors caused by insufficient repository understanding. Recent methods attempt to mitigate this limitation through pre-repair repository exploration; however, their fix-driven strategies expl...