Large language models (LLMs) often fail to utilise the context given in the input, resulting in hallucinations or a fallback to parametric knowledge. Context augmentation methods aim to improve model performance by modifying the input, from inserting filler tokens that add computational time to adding relevant content...
Haeun Yu, Geert Heyman, Pepa Atanasova et al.· 0 citations
Chain-of-thought (CoT) can sound plausible yet be unfaithful to the model's underlying reasoning. Most prior work probes CoT faithfulness through input--output behavior or input attributions, leaving internal computation largely underexplored. We instead cast faithfulness as internal concept grounding: Does a large lan...
Qianli Wang, Yi-Long Wang, Dennis Wei et al.· arXiv.org· 1 citation
This work proposes a novel rank-2 projection subspace that disentangles PK and CK contributions more accurately and uses it for the first multi-step analysis of knowledge interactions across longer NLE sequences, revealing that hallucinated generations exhibit strong alignment with the PK direction, though context-fait...