Controlling an acting LLM agent means deciding, at each step of a long-horizon task, whether to trust its next action, abstain, or hand off to a human. That decision needs a per-step confidence signal predicting whether the current rollout will eventually succeed. The agent's own self-reported verbalized confidence is...
EdgeGen is a synthetic task generation framework that extracts compliance rules from an agent's specification and uses them to generate database-grounded edge-case tasks designed to violate these rules, and when combined with existing synthetic data generation techniques, enables agent improvement through finetuning an...
Harshavardhan Abichandani, Penny Chong, Jiyuan Shen et al.· 0 citations
CoHyDE is introduced, an iterative procedure that trains the dense encoder and the LLM rewriter as a single co-evolving system that improves over the strongest single-component baseline by +2.5 pp NDCG@5 on standard queries and +6.3 pp on held-out vague queries.
Vaishali Senthil, Ashutosh Hathidara, Sebastian Schreiber· arXiv.org· 1 citation
This work introduces ToolSense, an open-source LLM-powered diagnostic framework that takes any tool catalog as input and automatically generates three benchmarks: a Realistic Retrieval Benchmark (RRB) with queries at three ambiguity tiers, an MCQ probing benchmark, and a QA probing benchmark.
Ashutosh Hathidara, Sai Shruthi Sistla, Sebastian Schreiber et al.· arXiv.org· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.