The results characterize conditional trade-offs, not universal superiority, so an absolute tolerance can constrain decisions without ensuring proportional ranking preservation or deployment utility.
Abstract
When numerical predictions and ranked recommendations have task value, selection must expose their trade-offs. We examine selection consequences and incremental uncertainty value in biased matrix factorization. Across 35 configurations and four training-evidence levels on MovieLens 1M, CiaoDVD and Yelp–Tampa, validation root mean squared error (RMSE) and normalized discounted cumulative gain for the top ten recommendations (NDCG@10) select different configurations in eleven of twelve conditions. Fresh-seed confirmation shows positive ranking regret whenever selections differ, but only four intervals lie wholly above the declared absolute tolerance of 0.005. An uncertainty-aware (UA) rule minimizes RMSE subject to a paired-seed upper bound on validation ranking regret. A revision-stage mean-regret ablation removes only the uncertainty penalty. At 0.005 tolerance, choices differ in seven of sixteen conditions, and UA improves held-out ranking point estimates in six at higher global RMSE, with no uniform improvement across both endpoints. Four-margin reselection and validation-user resampling expose sensitivity. In the frozen prospective Epinions evaluation, 35 candidates are admissible and UA equals RMSE selection. All four absolute-margin tests pass, yet ranking losses represent 28.2–62.2% of reference NDCG. Expanded searches retain objective disagreement and finite budget limits. The results characterize conditional trade-offs, not universal superiority, so an absolute tolerance can constrain decisions without ensuring proportional ranking preservation or deployment utility.
Recommender Systems (RS) are typically evaluated using accuracy and ranking metrics, but these metrics do not show whether the distribution of predicted ratings resembles the distribution of observed ratings. This aspect is especially important in confidence-aware recommendation, where confidence calibration indicates...
J. Pires, Eduardo Ferreira da Silva, Mayki dos Santos Oliveira et al.· Brazilian Symposium on Multi...· 0 citations
Findings show that Hodge-theoretic structure is informative not only for aggregating collected preferences, but also for deciding which preferences to collect.
Eskander Bejaoui, M. O. Aoueileyine, R. Bouallègue· International Journal of Adv...· 0 citations
Two-stage recommenders first retrieve a small candidate set from a large catalog and then apply a more detailed model to rank only those items. A common offline test instead gives every system the same candidates so that their scoring rules can be compared fairly, but this fixed-candidate protocol may select a differen...
Teresa Zhang· Proceedings of the 20th ACM...· 0 citations
Two estimators based on the decomposition of clicks into examination and relevance are proposed, one of which corrects the IIPS bias using policy examination probability ratios and extends LE-IIPS to a doubly robust framework.
A controlled empirical study of two decision-oriented systems, TypeSafe Jev and OpenAI Decisions, for recommendation reranking, suggests that specialization for decision-making does not inherently guarantee superior recommendation quality or robustness.
Candidate-dependent routing lets a sequential recommender weight a user's past behaviors differently for each item it scores. We test whether this flexibility helps by adding a small residual routing module to a self-attentive baseline, in two versions that differ only in what sets the routing weights: the user's histo...
Han Chen, Ying-Rui Li· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.