2 papers indexed here
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
Semantic Association vs. Position: Which Information Do Attention Heads Prefer?
In large language models (LLMs), attention heads generally encode both semantic association information (SAI) and positional information (PI). This raises a fundamental interpretability question: Which does each attention head relatively prefer, SAI or PI? To address this question, based on theoretical analysis and der...