A Simple Complexity Lower Bound for Solving $Ax=b$
In this note, we provide a short and direct proof that approximately solving $Ax=b$ to relative error $\varepsilon$, where $A$ has condition number $\kappa$ and unrestricted dimension, requires $\Omega(\kappa\log(1/\varepsilon))$ matrix-vector multiplications in the worst case, even for randomized algorithms. This esse...