For every $N\ge 2$, we prove that the Bergman metric on the regular locus of a finite ball quotient $\mathbb{B}^N/\Gamma$, where $\Gamma\subset \mathrm{U}(N)$ is finite and fixed-point-free, is K\"ahler-Einstein if and only if $\Gamma$ is trivial. Consequently, if $\Omega$ is an $N$-dimensional normal Stein space with isolated singularities and compact, smooth, strongly pseudoconvex boundary admitting a real-algebraic CR realization, then the Bergman metric on $\Omega_{\mathrm{reg}}$ is K\"ahler-Einstein if and only if $\Omega$ is biholomorphic to $\mathbb{B}^N$. This proves an algebraic version of the Cheng-Huang-Xiao conjecture in every complex dimension $N\ge 2$.