Speculative Probing: LLM Monitoring at Speculative-Decoding Cost
It is found that the speculative-decoding module in recent LLMs can be repurposed for efficient high-quality classification by appending a trained soft prompt at the end of the target sequence, which can repurpose the speculative-decoding module into a sequence classifier.
Collin Zhang, Ting-Wei Zhang, Vitaly Shmatikov
· 0 citations