Two-Level Softmax Sampling Done Right: Correcting Bias from Size Imbalance and Dispersion
Two sampling methods are proposed, Size-Corrected 2LS (S-2LS) and Size- and Dispersion-Corrected 2LS (SD-2LS), which provide provably better softmax approximations with negligible to non-existent computational overhead.
Walid Bendada, Guillaume Salha-Galvan
· 0 citations