This work revisits Locality-Sensitive Hashing (LSH) and introduces SOCKET, a SOft Collision Kernel EsTimator that replaces hard bucket matches with probabilistic, similarity-aware aggregation, and reframes LSH from a candidate-generation mechanism into a principled scoring kernel for sparse attention.
Sahil Joshi, Agniva Chowdhury, W. Bellinger et al.· 3 citations
Tail-Aware Credit calibratiOn (TACO), a method that calibrates uniform credit assignment to suppress undesirable positive updates in RL, and improves training stability, supporting sustained performance gains in long-horizon RL.
HiF4 is established as the enabling format for end-to-end FP4 RL post-training, and Rollout Residual Quantization (Rollout-ResQ) is established as the activation-side mechanism that makes the gap to BF16 closable.
H. Mak, Shadan Golestan, H. Le et al.· arXiv.org· 0 citations
This work proposes Residual Fallback Quantization (RFQ), a lightweight activation reconstruction framework that supplements the primary ulta-low-bit activation representation with an auxiliary quantized residual pathway that improves activation fidelity while preserving the efficiency advantages of ultra-low-bit comput...