This work optimize a document suffix by discrete token search, starting from the answer to a single question and extending the objective to a set of passages, to multi-hop questions that follow from the planted fact, and to later turns of a conversation.
Large language models (LLMs) often fail to utilise the context given in the input, resulting in hallucinations or a fallback to parametric knowledge. Context augmentation methods aim to improve model performance by modifying the input, from inserting filler tokens that add computational time to adding relevant content...
Haeun Yu, Geert Heyman, Pepa Atanasova et al.· 0 citations
Chain-of-Thought (CoT) faithfulness matters for monitoring, explaining and safely deploying large language models, yet it has been studied mostly in English. Languages differ in level, the amount of text available to learn from, so models know less in lower-resource languages, which may change how much they rely on the...
Lucas Resck, Jing-Yi Sun, Qianli Wang et al.· 0 citations
Chain-of-thought (CoT) can sound plausible yet be unfaithful to the model's underlying reasoning. Most prior work probes CoT faithfulness through input--output behavior or input attributions, leaving internal computation largely underexplored. We instead cast faithfulness as internal concept grounding: Does a large lan...
Qianli Wang, Yi-Long Wang, Dennis Wei et al.· arXiv.org· 1 citation
This work proposes a novel rank-2 projection subspace that disentangles PK and CK contributions more accurately and uses it for the first multi-step analysis of knowledge interactions across longer NLE sequences, revealing that hallucinated generations exhibit strong alignment with the PK direction, though context-fait...