LLM-agent evaluation reports behavioral scores but leaves the identity of the deployed runtime stack unreported. A tampered deployment that preserves evaluated outputs passes every output-only evaluator. We introduce DIB-Bench, a benchmark that measures this deployment-integrity blindness across 7,560 scored examples,...
Jian-Chang Su, Da-Wei Xiang, Yi-Fan Zhang et al.· 0 citations
Retrieval-augmented fact-checkers often receive a reliability label, such as HIGH or LOW trust, for each evidence source. These labels should adjust the model's confidence and its decision to search for more evidence, while the verdict should follow the evidence content. We introduce TrustSwap, a counterfactual test th...
Jian-Chang Su, Yi-Wei Yang, Wei Zhang· 0 citations
Cached Diverse Influence Selection (CDIS), which recomputes gradient features for the top-ranked fraction of candidates under the stale scores, fits an affine calibration on the recomputed examples, and selects the final subset under source and length quotas.
Jian-Chang Su, Yi-Fan Zhang, Wei Zhang· 0 citations
FactorDLM is introduced, a training-free decoder that represents finite-domain relations as a factor graph and, at each denoising step, conditions the model's mean-field prediction on that graph exactly by variable elimination, increasing decoding cost exponentially with the induced width of the constraint graph.
Measurements taken at the start of a fine-tuning run can bound, for each protected fact, the probability that the run makes the model forget it, and first-order models of fine-tuning can thus bound forgetting on the facts whose response coefficients change by less than their distance to the boundary.
Jian-Chang Su, Wei Zhang· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.