This work introduces a Hierarchical Dirichlet Process Gaussian Mixture Model over behavior embeddings, which shares components across domains while allowing domain-specific mixing weights, and uses the posterior predictive to retrieve relevant behaviors for a query, and shows that the Bayesian Cheatsheet is an adaptive...
Keshav Ramji, Tahira Naseem, R. Astudillo· 0 citations
Looped Transformers apply the same layers several times per token, adding compute to improve quality without more parameters. Each recursion, however, writes its own key-value cache, so memory still grows with compute. Inference-time techniques can shrink this cache at a cost in quality. We pretrain looped language mod...
Giovanni Monea, Keshav Ramji, Yousef El-Kurdi et al.· 0 citations
Hypernetworks have recently shown success in dynamically adapting the parameters of Large Language Models (LLMs) at runtime based on signals such as task descriptions or additional demostrations. Here we ask: how much adaptation signal can be obtained using only the input query to an LLM?. To answer this, we study quer...
A. Khan, Keshav Ramji, Tahira Naseem et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.