Quang Minh Nguyen

Whether LLMs Can Navigate Beliefs and Facts Depends on How You Phrase It

It is shown how fact-checking, a generally desirable behavior, can interfere with belief tracking in LLMs and how suppressing this attention at decoding time recovers accuracy only partially and only in some models, calling for future work on intervention methods.