Constrained nonconvex optimization has seen increasing application in modern machine learning, such as safe LLM alignment/finetuning, transfer learning and rank-constrained continual learning. The commonly used approach is Projected SGD which requires projection at every iteration. However, projection onto a functional...
Yu-Yang Deng, Mohammadreza M. Kalan, Eitan J. Neugut et al.· 0 citations
This work describes the distribution-free minimax rate of excess abstention risk, up to logarithmic factors, in terms of the complexity of the hypothesis class and the sample size and introduces surrogate-loss formulations to make the framework amenable to computation with models such as neural networks.
Mohammadreza M. Kalan, Yu-Yang Deng, Sanaz Hamidi· 1 citation
We consider the problem of noisy gradient-free minimization of the k-th order partial derivative of a $\beta$-H{\"o}lder function supported on a d-dimensional cube. We show that T ^{($\beta$+d+k)/(2$\beta$+d)} log(T )^{(\beta-k)/(2\beta+d)} is a non-asymptotic minimax rate of the T step cumulative regret for all $\beta...
Théo Paquier, A. Tsybakov, F. Portier et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.