In-Batch Negatives Can Silently Cripple LLM-Encoded Sequential Recommenders

Abstract

LLM-based sequential recommenders increasingly train user/item encoders with in-batch InfoNCE negatives inherited from contrastive learning—convenient, but it silently caps the negative pool at batch size rather than catalog size whenever encoder cost bounds the batch. We report an early finding: a 15-negative in-batch objective stalled validation NDCG@10 at 0.035 for dozens of epochs, well below a strong ID-based baseline, while full-catalog softmax cross-entropy plus a whitened, mixture-of-experts item encoder nearly doubled it and closed the gap with a state-of-the-art semantic recommender to a near-tie. We describe this fix and the open puzzle it exposes: ablations rule out item-text semantics, auxiliary losses, and temporal windowing as the source of the remaining gain, and a completed sensitivity sweep instead points to contrastive temperature as the dominant lever, whose optimal value reverses once the sampling objective is corrected.