This study reveals that the Deepseek-R1-Distill series models are subject to significant fluctuations caused by various factors, making their claimed performance improvements difficult to reproduce reliably.
Yong-Fu Zhu, Lin Sun, Jinzhu Wu et al.· arXiv.org· 4 citations· ⚡2
Benchmark saturation and data contamination increasingly obscure genuine scientific reasoning in frontier LLMs. We introduce \textsc{ScienceArena}, an olympiad-style benchmark from thirteen public science competitions in physics, chemistry, and biology, including IPhO and IChO 2025--2026, IBO 2023, USAPhO 2026, and USN...
Guangxiang Zhao, Qi-Long Shi, Xusen Xiao et al.· 0 citations
RealClawBench is introduced, a live benchmark framework built from real OpenClaw sessions to capture the distribution, diversity, and real-world difficulty of deployed agent use and provides a practical path toward benchmarks that better measure agent capability in actual use.
Zongwei Lv, Zhewen Tan, Yao-Ming Li et al.· arXiv.org· 1 citation· ⚡1
This work introduces ArborMem, an online memory framework that represents a long-running conversation as a navigable forest of interaction states that outperforms the strongest baselines on three established benchmarks and introduces BranchMemEval, a controlled diagnostic benchmark for interleaved and resumable interac...
Zongwei Lv, Yue-Meng Xu, Yilun Yao et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
