Controlling Risks of AI in Chemical Science with Agents
SciGuard is proposed, an agent-based guardrail that employs large language models, tools and external knowledge to assess and control risks in scientific AI interactions, and achieves a state-of-the-art harmlessness score on red-teaming queries, while maintaining high performance on benign tasks, without sacrificing sc...
Jiyan He, Hao-Xiang Guan, Weitao Feng et al.
· AI for Science · 26 citations
· ⚡4