Sparse autoencoders (SAEs) decompose LLM activations into sparse dictionary atoms, so that each distinct concept gets its own feature. One recurring behavior complicates this premise: feature absorption, in which a parent concept and its children--fruit and {apple, banana, pear}, say--collapse into a shared family dire...
Jin-Yuan Zhang, Peng-Ji He, Yin Yuan et al.· 0 citations
Physics-Informed Neural Networks (PINNs) frequently fail on stiff or advection-dominated PDEs, and two recent accounts offer competing remedies: switching from FP32 to FP64 to repair an L-BFGS stopping artifact, or replacing the MLP with a state-space-model (SSM) backbone plus sub-sequence alignment to counter architec...
Jin-Yuan Zhang, Peng-Ji He, He-Long Hu et al.· 0 citations
The main lesson is diagnostic: attention-level ICL proxies earn their place as training targets only after validation against behavioural gaps, and how far they can be trusted once it is optimised is asked.
Jin-Yuan Zhang, Peng-Ji He, He-Long Hu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.