Controlling Risks of AI in Chemical Science with Agents
SciGuard is proposed, an agent-based guardrail that employs large language models, tools and external knowledge to assess and control risks in scientific AI interactions, and achieves a state-of-the-art harmlessness score on red-teaming queries, while maintaining high performance on benign tasks, without sacrificing sc...