Background: Choosing an agent controller and deciding which component to improve require different evidence. A replacement can change both the decision rule and what the agent can do after that decision. Objectives: We ask what existing replacement comparisons establish about task benefit and about attribution to local...
Web agents and answer engines fetch pages at question time, and liveness checks of cited sources treat an HTTP 200 response as success. We measure how often that assumption fails and what it does to measurements built on fetched pages. We fetched 10,307 URLs in three days with a polite static client and a rendered brow...
Background. A component replacement in a language-model agent changes an execution trajectory, potentially altering later observations, resource use, and recovery opportunities. Different evidence is needed to assess its task-level benefit and the contribution of local decision quality. Methods. This critical scoping r...
This critical narrative review examines what these evaluations estimate and which conclusions their designs support and provides a methodological synthesis and a reporting checklist to help align claims about tools and skills with the comparisons their evaluation designs support.
Scaling model-generated data is usually viewed as improving distillation: more examples should increase coverage, reduce noise, and produce stronger students. We show a second effect: larger datasets can make subtle teacher-specific signals easier to detect in the trained student, even when examples are off-task and ne...
Zhichen Dong, Zhi-Xuan Liu, Yuanjiu Fan, et al.· 1 citation
A semantic-entropy-based method, using task uncertainty to guide prompt optimization, which requires no training, works with black-box models, and integrates easily into existing prompt optimizers.
Shuyang Zhang, Zhixuan Liu, Zhichen Dong et al.· Annual Meeting of the Associ...· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.