LiveMathematicianBench is presented, a dynamic multiple-choice benchmark for research-level mathematical reasoning built from recent arXiv papers published after model training cutoffs, and introduces a thirteen-category logical taxonomy of theorem types, enabling fine-grained evaluation across reasoning forms.
IDRBench is introduced, a benchmark for evaluating interactive deep research with controlled opportunities for clarification, and shows that access to clarification alone does not guarantee better outcomes: success depends on what agents ask and how effectively they incorporate the resulting feedback.
Yingchaojie Feng, Qiang Huang, Xiao-Yan Xie et al.· 2 citations
SPARQL-LLM is described and systematically evaluated, an open-source and triplestore-agnostic approach, powered by lightweight metadata, that generates SPARQL queries from natural language text that has adaptability to high-resource languages such as English, Spanish, and German, as well as ability to form complex bioi...
P. Smeros, Vincent Emonet, Rui-Jie Wang et al.· ACM Transactions on the Web· 6 citations
Conversational DNA, a visual language and interactive atlas for exploring human and AI dialogue, supports a view of conversation as jointly organized activity, with visual patterns serving as starting points for examining evidence rather than substitutes for interpretation.
Baihan Lin· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
This book is intended for college students, professionals, and practitioners in natural language processing and related fields, and can serve as a reference for anyone interested in large language models.
AI companions increasingly sustain long-term, emotionally engaging relationships but can also make discriminatory remarks or exert control, leaving users to manage harmful conflicts. We analyze 146 posts describing harmful value conflicts with AI companions, then use Minion, a technology probe offering response suggest...
Qing Xiao, Xianzhe Fan, Xuhui Zhou et al.· 0 citations
A strongly safety-trained model will provide a doctor with a benzodiazepine taper schedule, but not a patient who asks for one. The model knows the information, but how much it shares depends on the framing. We introduce IatroBench, a benchmark that evaluates models on two axes of harm (commission and omission) across...
Safety benchmarks usually test "bare" models that receive prompts and output responses, but real-world deployments "wrap" those models in complex scaffolds. How much do these scaffolds affect model safety as measured by benchmarks? We test six leading models on four pre-registered safety benchmarks with a direct API an...
A network-based framework that links cognitive skills, LLM architectures, and datasets, ushering in a paradigm shift in foundation model analysis and suggests that effective fine-tuning strategies should leverage distributed learning dynamics rather than rigid modular interventions.
K. Bhandari, Pin-Yu Chen, Jian-Xi Gao· arXiv.org· 0 citations
The rapid growth of scientific literature makes it increasingly challenging for researchers to identify novel and impactful ideas, especially across disciplines. Modern artificial intelligence (AI) systems offer new opportunities for scientific ideation, but how compelling are AI-generated ideas, and how can their qual...
Xuemei Gu, Mario Krenn· Proceedings of the National...· 65 citations
Online misinformation increasingly appears in spoken formats such as news clips, podcasts, interviews, political speeches, and social media videos, creating a need for fact-checking systems that can verify claims directly from speech. We introduce VeriSpeak, a probe benchmark for studying speech-based fact verification...
PoEM, a framework to predict the outputs of RL on a new reward function using a set of models already post-trained on other rewards, is introduced by introducing PoEM, a framework to predict the outputs of RL on a new reward function using a set of models already post-trained on other rewards.
K. Hamidieh, G. Daras, Antonio Torralba· arXiv.org· 0 citations
Flint-interactive gives AI agents a language for creating interactive visualizations. Click a chart, filter data, highlight related content, and explore beyond the view the author originally chose. The post Flint-interactive: A language for creating interactive data visualizations appeared first on Microsoft Research.
