Author

Jaekyun Moon

h-index 5 View on Semantic Scholar

We have 1 of 12 papers

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

Jul 2026

ELSAA: Efficient Low-Rank and Sparse Attention Approximation for Training Transformers

ELSAA is proposed, an efficient low-rank and sparse approximation of attention that gives a practical framework for constructing low-rank and sparse attention outputs without materializing the full quadratic score matrix, aiming to enable longer-context training while preserving both sharp token-level interactions and...

Mahdi Heidari, Mohammad Mahdi Rahimi, Jaekyun Moon · arXiv.org · 0 citations