Methods for assessing language-model agents include rule checkers over logs, analyses of skill coverage and composition, support checkers, prefix monitors, execution gates, and rare-event estimators. Each observes a different part of a run and makes a claim of a different strength, and no common account says how these...
This work introduces PrefixGuard, a neuro-symbolic method that uses learned events to connect online failure warning with finite-state execution diagnosis with finite-state execution diagnosis in large language model (LLM) agents.
Xin-Miao Huang, Jinwei Hu, Qi-Song He et al.· 1 citation
It is found that a simple random-without-replacement routing policy, which lets each agent debate with two distinct and newly sampled peers at every round, provides a surprisingly strong baseline and consistently improves the accuracy-cost trade-off of sparse MAD.
Bo-Xuan Wang, Zhuo-Yun Li, Xiao-Wei Huang et al.· 0 citations
Cross-modal evidence noncommutativity is revealed: the same evidence can lead to different judgments when its order changes, and placing perceptual evidence later can increase the model's reliance on its content.
Zhuo-Yun Li, Bo-Xuan Wang, Xiao-Wei Huang et al.· 0 citations
This paper introduces SCOPE (Sequential Conformal OOD Probing and Evaluation), a framework that selects a readable hidden layer, constructs a conformal gate with IND calibration, and uses a supermartingale e-process to certify persistent service-boundary evidence.
FragileFlow is introduced, a plug-in regularizer that uses a calibrated margin buffer to identify correct-but-fragile predictions and organize their off-class probability mass into a class-wise vulnerable-risk matrix and provides the first PAC-Bayes upper bound for this margin-aware error-flow object.
Zhuo-Yun Li, Bo-Xuan Wang, Jinwei Hu et al.· arXiv.org· 4 citations
The Reasoning Backroom is established as a general AI provenance problem whose audit requires intervention because of a pervasive provenance failure in multi-agent systems.
Jinwei Hu, Yi Qi, Xin-Miao Huang et al.· arXiv.org· 0 citations
This work proposes Logical Graph Uncertainty (LGU), a framework that explicitly models implication and incompatibility among answers, and ranks first on average among existing uncertainty measures.