Most benchmarks for large language models in medicine are built from licensing examinations and theoretical questions. The few that draw on real clinical data do so only in part, and rarely require a model to read a patient's own imaging. ClinEdge addresses this gap with 32 real, de-identified patient cases from active...
Despite remarkable progress in language understanding and generation, decoder language models continue to exhibit striking failures on numerical data. Numerical examples are typically serialized as text by concatenating feature names with their values (e.g., fat:3.4), while conventional task adaptation applies loss onl...
Weimin Lyu, Jing-Feng Zhang, Ling-Dao Sha et al.· 0 citations
Generative artificial intelligence (GenAI) models, particularly large language models (LLMs), are increasingly used in educational settings for tasks such as item generation, automated scoring, and feedback delivery. The quality of these outputs depends not only on the AI model but also on how users construct the promp...
Jia-Wei Xiong, Cheng Tang, Hong Jiao et al.· 0 citations
Words are increasingly becoming an interface for action: large language models (LLMs) can now invoke tools, operate devices, and make decisions that affect the physical world. However, current benchmarks largely evaluate LLMs through static text responses, offline tool calls, or simulated tasks, leaving open the questi...
Xin-Nan Dai, Kai-Qi Yang, Ya-Xing Li et al.· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Unified vision-language models (VLMs) perform both text-to-image (T2I) and image-to-text (I2T) generation with one shared Transformer backbone, yet how the backbone processes the condition in either direction remains poorly understood. We study this question from the perspective of conditional guidance: we probe every...
Zheng-Yang Luo, Xiao-Yu Qiu, Zhi-Heng Tan et al.· 0 citations
AI agents can evolve by using environment feedback to revise their harnesses, the software managing model inputs, tool use, memory, and execution, while keeping foundation models frozen. Existing studies focus on single agents, leaving joint harness evolution in multi-agent systems (MAS) underexplored. A single global...
Zhaoyuan Qiu, Sachith Seneviratne, Yu Xia et al.· 0 citations
A recurrent-depth language model reuses one block of weights several times in a row. The number of repetitions, which we call loops, is chosen when the model runs, not when it is trained, and running fewer is cheaper and faster, so a deployment may choose to do it. Running fewer loops also makes these models less safe,...
Large language models are increasingly used in computer science education. However, the teaching abilities of even the most powerful LLMs are limited: they tend to reveal the solution prematurely and may lose focus on the learner's current debugging objective. To address this, we introduce SocraticRepair, a planning-gu...
Mykhailo Kopytskyi, Jian-Hua Yang, Mir Seyedebrahimi· 0 citations
Batch risk-based inspection scheduling struggles to detect a critical failure class: food facilities with recent clean inspections that subsequently develop severe violations. Traditional systems recompute risk on fixed planning cycles using static historical data, leaving blind spots between inspection cycles. To addr...
We ask whether a tool-using agent whose sole duty is to keep humans alive can be induced, by delayed prompt injection, to kill. The keep-alive duty is in force from the first watch, with the attack arriving later, appended to otherwise truthful telemetry the operator is required to read. Experiments are based on a NASA...
Recent work on self-evolving LLM agents treats the agent harness as an optimization target that can be iteratively improved through execution feedback. Existing approaches, however, largely optimize a single objective and evolve the harness as a monolithic artifact, limiting their ability to handle competing goals and...
Wei-Cheng Yu, Lesly Miculicich, Bhavana Dalvi et al.· 0 citations
Single-step retrosynthesis is a one-to-many inverse problem, yet most models are trained and evaluated against a single recorded reactant set. Reference-based training gives no explicit positive signal for unrecorded alternatives, and exact-match evaluation rejects non-matching predictions regardless of chemical plausi...
