The pursuit of autonomously self-improving models has attracted growing interest in the era of large-scale foundation models. Drawing inspiration from the concept of"enlightenment"or"aha moment"in human brain, we hypothesize that large models exhibit an analogous enlightenment phenomenon-a latent capacity for sudden capability boost. Then, we propose Enlightenment, a novel training-free post-tuning paradigm for large-scale models. Our approach modifies shortcuts for key modules/layers without weight updates, while existing training-free ones predominantly manipulate attention weights. We introduce two architecture-specific instantiations: i) For large language models, we propose attention head-mixing shortcuts that recalibrate attention weights by linking the initial attention head's output to all other target heads, modulated by an adaptive scaling factor initialization strategy. ii) For vision-language models, we apply a lightweight scalar-modulated factor to residual connections in the decoder layers, regulating information flow. Extensive experiments show that Enlightenment efficiently unlocks the latent potential of pre-trained networks, yielding remarkable performance improvements across diverse benchmarks and models.
By overcoming the longstanding memory and scalability barriers, RPG serves as a critical advance in ‘ AI generating AI ’, potentially enabling efficient weight generation at scales previously deemed infeasible.
Kai Wang, Dongwen Tang, Wangbo Zhao et al.· Neural Information Processin...· 7 citations· ⚡1
LingBot-Video is presented, a DiT-based video pretraining paradigm specifically tailored for embodied intelligence, and is contributed as the inaugural large-scale, open-source MoE video foundation model to the community, in a pioneering effort to bridge digital creativity and physical actuation.
Shuailei Ma, Jiaqi Liao, Xinyang Wang et al.· 5 citations
To mitigate a critical imbalance during the exploration-and-learning process, this work approaches head-tail re-balance during the exploration-and-learning process from two perspectives: distribution-reshaping and trajectory-resampling.
Xin Guo, Zhiheng Xi, Yiwen Ding et al.· Annual Meeting of the Associ...· 1 citation
This work introduces Generative Pretrained Controllers (GPC), which leverage tokenization and next-token modeling to create general-purpose, reusable generative controllers from large-scale motion datasets.
Yi Shi, Yifeng Jiang, Chen Tessler et al.· International Conference on...· 1 citation
Text-to-image diffusion models can synthesize unsafe content on demand, which makes concept erasure an active safety problem. Existing training-free methods all intervene on the model's content pathway: they steer the predicted noise, project the text embedding, or rewrite cross-attention weights, controlling what the model paints. Yet a visual concept is carried not only by content but by structure, the way spatial parts bind into a coherent whole, which in diffusion U-Nets is governed by self-attention. We introduce SADE (Self-Attention Disruption Erasure), a training-free method that erases a concept by disrupting its structural assembly rather than its content: it reads a spatial saliency for the concept from the self-attention features, then down-weights the mutual attention among salient positions so the concept cannot form. The operator changes no weights, adds one strength parameter, and provably reduces to ordinary generation when the target concept is absent. On I2P, SADE matches state-of-the-art training-free erasure while preserving COCO image quality best among the compared methods, and its prompt-independent saliency resists text-based attacks. Existing methods stop the model from painting the concept; SADE lets it paint but keeps the pieces from coming together.
Xiaochuan Jin, Zheling Meng, Bo Peng et al.· 2026 6th International Confe...· 0 citations
The Robust-WAM is a general post-training method for video-generation-based WAMs that preserves the VAE-based generative path and adds a lightweight semantic foresight alignment objective on the action stream to retain the large-scale VGM pretraining while grounding actions in appearance-invariant dynamics.
Haodong Yan, Junfeng Li, Junjie He et al.· 0 citations