Prompt learning is a popular method for adapting foundation models, but learned prompts are typically task-specific and fail to generalize to new classes, domains, or compositions of tasks. In this paper, we introduce a Diffusion Meta-Prompt (DMP) model , a framework that models the distribution of learned prompts usin...
Deepak Sridhar, Yi Li, K. Bhardwaj et al.· 1 citation
Multimodal models and agents have emerged to enable AI systems to learn from and interact with the real world more like humans by integrating information across multiple modalities. However, enabling multimodal agents to continually acquire useful information from complex, heterogeneous multimodal data remains a signif...
Ming-Hao Guo, Qing-Yue Jiao, Xiu-Jin Liu et al.· 0 citations
On-policy distillation (OPD) uses teacher correction on student-generated responses. Full-vocabulary correction can provide important corrections even for tokens that the student assigns low probability, but backpropagating through all token logits becomes memory-intensive for long sequences. Existing memory-saving app...
Yongliang Miao, Shuang Liu, Yan-Guang Liu et al.· arXiv.org· 0 citations
This work proposes RewardExplainer, a framework that obtains feedback from the target reward model through counterfactual rewriting and uses this feedback to further optimize the explainer, enabling the explainer to more faithfully capture the target RM's scoring preferences and sensitive behaviors than single-pass gen...
Jing-Yi He, Ni-Er Wu, Shuang Liu et al.· arXiv.org· 0 citations
Reward models score responses from large language models (LLMs) and guide LLM training toward human preferences. However, reward models can favor superficial attributes such as length or confidence, leading LLMs to produce higher-scoring but not more correct responses. Existing mitigation methods either retrain the rew...
Shuang Liu, Yongliang Miao, Yan-Guang Liu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.