How to"train"a surgeon? Benchmarking generalist agents in surgical scene understanding
SurgAgentGym is introduced, an agentic environment where general-purpose MLLM agents can use bounded knowledge search over SurgWiki and general-purpose visual grounding for surgical tasks and shows that agentic inference narrows the gap mainly for perception-heavy tasks, while fine-grained localization and expert-style...