Skip to content

Author

Avni Mittal

2 papers indexed here

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

#artificial intelligence Preprint Nov 2026

Sonar-MoE: In-Backward Expert Updates for Memory-Efficient Mixture-of-Experts Training

Most parameters of a Mixture-of-Experts (MoE) model belong to its experts, and standard training keeps every expert gradient in GPU memory until the optimizer step. This memory grows with the number and size of the experts, so it rises as MoE models scale. We introduce Sonar-MoE, which updates each expert tensor during...

Avni Mittal, Avinash Anand, Ashutosh Kumar et al. · 0 citations
Preprint Aug 2026

TextNCA: Neural Cellular Automata for Language Modeling via Hierarchical Local Attention

The work positions the work as a controlled reading of which parts of NCA-style computation carry the weight in language modelling, and examines a hierarchical variant of the Neural Cellular Automaton that cascades three stages with windows.

Avni Mittal, Avinash Anand, Ashutosh Kumar et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.