Transformers and state-space models (SSMs) are the two dominant families of sequence models, and a central open question is how far the analytical knowledge built for transformers transfers to SSMs. We address this through the lens of layer importance which underpins compression, selective fine-tuning, and interpretabi...
Istabrak Abbes, Nizar Islah, Irina Rish et al.· 0 citations
Metagenomic sequencing has uncovered billions of proteins from uncultured microorganisms, vastly expanding the known protein space. Yet most remain functionally inaccessible because existing annotation methods depend on close homologs or accurate structure predictions. Here, we show that protein language models (pLMs)...
Lola Le Breton, David Heurtel-Depeiges, Douglas C. Millar et al.· bioRxiv· 0 citations
Motivation Antimicrobial resistance (AMR) has been identified as a top global public health threat. Accurate AMR phenotype prediction from whole-genome sequencing data is an essential tool for accelerating clinical decision-making and mitigating resistance spread. Although many previous works have explored the use of t...
Alex Aselstyne, E. Karthik, Meriem El Azami et al.· bioRxiv· 0 citations
A novel architecture incorporating an explicit private working memory is proposed and it is demonstrated that this mechanism restores consistency with a fixed hidden state, establishing private state as a necessary component for PSIT-capable language agents.
Davide Baldelli, Alipanah Parviz, A. Zouaq et al.· arXiv.org· 2 citations
This work uses encoders producing sparse latents in training Sparse Koopman Autoencoders without basin labels or other regime annotations to identify sparse latents and their corresponding supports as label-free, interpretable regime variables for Koopman learning in nonlinear systems with multiple local dynamical laws...