HARISSA: Inference-Time Self-Checks for Efficient and Safe Local Language Model Deployment
HARISSA fine-tunes the model so that both states predict correctness, then makes both decisions with one policy that cascades through the ways of answering from cheapest to most expensive, skipping a way the prefill state predicts will fail and deferring the query when the answer it stops with is predicted wrong.
Kenan Alkiek, Moontae Lee, David Jurgens et al.
· 0 citations