Continual adaptation of language models can change their output distribution on prompts learned earlier, while retaining every old prompt-answer pair may be undesirable or impossible. We study condition-anchored generative distillation (CAGD): retain a small set of old prompts, use a frozen previous model to reconstruc...
Results indicate that multi-scale Inception encoding and auxiliary supervision are the most important components, while the PatchX interaction module provides no consistent additional gain under the current setting.
Tian-Chang Xie, Hai-Ling Wang, Wei-Guang Wang et al.· IEEE Photonics Journal· 0 citations
The Personal History Memory Mechanism is introduced, which replaces random perturbation with weighted historical experience to enhance the directional search capability of the algorithm, and a greedy selection strategy is embedded to ensure the monotonically non-deteriorating quality of population solutions.