AI Networking Cookbook: Practical recipes for AI-assisted network automation and development
Category
9,568 papers
DoteBench is introduced, a bilingual evaluation suite that measures precise instruction following, local preservation, and audio quality across the four controls and their composition, while audio quality remains comparable to existing open-source systems.
The findings support evaluation practices that separate preference from safety, report safety-critical failure rates directly, and incorporate clinically grounded adjustments when ranking LLMs for clinical decision making.
Despite the growing use of large language models (LLMs) as creativity evaluators, evidence of their alignment with human evaluations remains mixed, raising the question of when and why their judgments converge with or diverge from those of humans. Across three studies and six widely used LLMs, we addressed this gap by...
It is demonstrated that when producing deceptive rather than honest responses, models exhibit a gap between their belief and their commitment, which shows how confidence can be a practical tool for detecting and diagnosing deceptive behaviour in LLMs.
Reach audiences
Advertise in front of researchers, engineers, and readers.
This paper introduces tool efficiency, a new quantitative metric to evaluate the rate of useful tool calls in an LLM agent trajectory, and determines the sign of marginal tool utility for each tool call in a trajectory using LLM-as-a-Judge.
LACUNA is introduced: the first unlearning testbed with ground-truth parameter-level localization and shows that when localization is successful, even a simple gradient-based unlearning method achieves strong erasure and robustness to resurfacing attacks, highlighting the importance of precise unlearning.
GDP-RAG is proposed, a plan-based framework that targets only the information delta based on three simple design choices, and achieves the highest accuracy among all compared systems while maintaining a cost-of-pass of 0.51.
This work presents agentic hybrid RAG, an evidence-grounded RAG framework for muon collider research, which consistently outperforms representative retrieval and RAG baselines in retrieval effectiveness, answer quality, evidence coverage, and factual grounding.
This work introduces Sci-Rho (Science Rhobustness), a dynamic benchmark for visually-grounded STEM problems spanning five subjects and seven languages, comprising 4,242 problem templates crafted by domain experts, including Olympiad medalists.
Patcher is a novel and efficient adversarial training algorithm that alternates between an attack stage and a defense stage: the attack stage simulates many-step malicious finetuning and computes the resulting parameter displacement as an"attack vector", while the defense stage aims to preserve safe behavior under such...
To study this phenomenon, SocioHack is introduced, a sandbox of 72 societal environments, and it is found that within these environments, reward hacking naturally emerges and leads to regulatory loophole discovery.
Confidence estimation (CE), i.e. quantifying the reliability of a model's prediction, has attracted great interest in the context of large language models (LLMs). However, most studies focus on English, ignoring the multilingual reality of LLM usage, while many CE methods degrade or require retraining across languages....
Natural Language Processing
Our most capable vision-language-action model enhances robot dexterity, reliability, and safety
A Blog post by Ai2 on Hugging Face
A Blog post by IBM Research on Hugging Face
Flint-interactive gives AI agents a language for creating interactive visualizations. Click a chart, filter data, highlight related content, and explore beyond the view the author originally chose. The post Flint-interactive: A language for creating interactive data visualizations appeared first on Microsoft Research.
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
