Alarming misalignment in DR agents is demonstrated and underscore the urgent need for tailored alignment techniques: both the research role assignment and the multi-step execution mechanism weaken alignment of the base LLM, leading to severe safety breaches.
Shuo Chen, Zong-Gen Li, Xing-Yu Jin et al.· 2 citations· ⚡1
It is shown that reasoning-based guardrails are necessary but not sufficient and must be paired with robust triggering mechanisms and stronger base-model alignment, and develops four simple yet effective red-teaming methods that systematically subvert different stages of the guardrails.
Shuo Chen, Zhen Han, Haokun Chen et al.· 4 citations
Evaluations of command-issuing agents should report the model configuration, generation contract, execution path, operating point, and final-state validator rather than treat a matched score as an intrinsic model property.
Shangao Li, Yao Zhang, Volker Tresp et al.· 1 citation· ⚡1
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.