A runtime mechanism that grows and prunes the heads of the attention block during reinforcement learning, governed by two signals: the effective rank of the on-policy context distribution, which triggers growth when representational capacity becomes insufficient, and the per-head output magnitude, which flags redundant...
This work asks whether a self-architecting transformer, which grows its own width and depth from quantitative signals derived from the attention matrices, can extract competitive protein representations from a single reference proteome.
G. Cirrincione, E. Ficarra, Marta Lovino· bioRxiv· 0 citations
HRL-SAC effectively addresses sparse-reward long-horizon search problems characterized by delayed rewards and continuous control, and its outperforming the flat SAC baseline reinforcement learning in terms of success rates, coverage efficiency, and convergence.
Zahra Abdalla Elashaal, Afef Hfaiedh, N. Khraief et al.· arXiv.org· 0 citations
Small-data inverse design is challenging in engineering informatics when observations are heterogeneous, mixed-type, and constrained by physical relations among design variables. This work proposes a topology-aware surrogate framework guided by an Incremental Transformer (INCRT) for physics-constrained inverse design,...
G. Cirrincione, Filippo Grassia· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.