LLMs are increasingly applied to cybersecurity workflows, where they are expected to translate analysts'intent into tool invocations. However, existing evaluations focus on knowledge-based assessments or end-to-end agentic tasks, and do not directly measure LLMs'ability to generate executable commands for real-world cy...
Peng-Fei Li, Naufal Suryanto, Si-Cheng Zhang et al.· 0 citations
SAFIRE is introduced, a large-scale benchmark for fire-smoke understanding in MLLMs, comprising 83K captioned images from 20 scenarios and 193K multiple-choice VQA (MCVQA) generated from a 9.7K-image subset, spanning 10 evaluation dimensions from basic perception to higher-order reasoning.
Peng-Fei Li, Naufal Suryanto, Si-Cheng Zhang et al.· 0 citations
Text-to-image (T2I) generation has achieved remarkable progress in recent years. However, existing research has largely focused on English-only settings, leaving cross-lingual performance gaps and language-specific effects insufficiently explored. To fill this gap, we introduce LingT2I, a benchmark covering 10 widely u...
Si-Cheng Zhang, Zhong-Hao Yan, Bin-Zhu Xie et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.