Self-attention gives LLMs fine-grained, query-dependent access to context, but dense token interactions incur quadratic prefill cost and a key--value cache growing with context length. Research thus spans explicit-memory compression, sparse access, recurrent state construction, structured state dynamics, and heterogene...
Zhen-Tao Tan, Jing-Yi Shen, Yan-Bo Li et al.· 0 citations
LucaCell, a sequence-centric foundation model that represents genes through pre-trained mRNA sequence embeddings rather than static gene annotations, is presented, showing that sequence-informed gene representation can improve the generalization of single-cell foundation models across species, data types, and predictiv...
Yan Sun, Yong He, Min-Si Ren et al.· bioRxiv· 0 citations
This work introduces two exact, gradient-equivalent corrections: LogitTree, a segmented K-forward traversal, and a packed 4D attention mask, and proposes SDCC (Self-Distillation for Conditioning Consistency), a single-backward-pass variational relaxation.
J. Zinco, Xun-Jie Zhu, Shen Huang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
