When an LLM judge scores an output, its score distribution retains uncertainty and disagreement information that is lost after scalar compression. We introduce JudgeMoE, a lightweight aggregator that assigns example-specific weights to cached judge score distributions and fuses them before computing a final score. A pr...
Yiqi Liu, Joseph James, Yang Wang et al.· 0 citations
When a language model refuses to answer a prompt, it is unclear whether the correct answer is erased from its internal representations, or merely suppressed at the output layer. We investigate this mechanism using a controlled withhold setting, which yields perfectly matched answering and refusal trajectories for bidir...
Yiqi Liu, Yang Wang, Songxin Wang et al.· 0 citations
This work proposes ToolScout, a source-aware routing method that uses query-side TF-IDF fingerprints flag source styles on which the fine-tuned retriever is likely to fail better than semantic or length-based proxies, giving a cheap signal for mismatch over a fixed tool corpus.
Yiqi Liu, Joseph James, Yang Wang et al.· 2 citations
This work presents the first systematic meta-evaluation of LLM-generated rubrics intrinsically by semantic similarity and extrinsically by score alignment with ground-truth rubrics, and meta-evaluate generated rubrics intrinsically and extrinsically by score alignment with ground-truth rubrics.