Best Subset Selection in Linear Regression: Fixed-Design Error Bounds and Insights for Random Designs
We study exact support recovery by best subset selection in linear regression under a fixed design. For a known support size $s$ and ambient dimension $d$, we derive a non-asymptotic upper bound on the probability that best subset selection fails to recover the true support. The bound is expressed through a determinist...