Disentangling Task Difficulty from Run-Level Failure in Agent Failure Prediction
Predicting whether an LLM agent will fail has emerged as a promising direction for supporting intervention during execution. Recent approaches report strong predictive performance, often with AUROC values between 0.85 and 0.94. However, predictors are typically trained by pooling runs from many tasks. We hypothesize th...
Mohsen EsfandyariDoulabi, L. Arkoh, B. Tadesse et al.
· 0 citations