Preprint

FlavourBench: Executable Culinary Reward Maps for Language Model Evaluation and Post-Training

J. Chen Erim Hayretci

Aug 2026 · 0 citations · 26 references

Computer Science

TL;DR

FlavourBench is introduced, which instead compiles dense answer maps from a versioned culinary environment and remains similar under alternative metrics, task filters, family weights, and three public Epicure checkpoints.

Abstract

We introduce FlavorBench: a benchmark for Compiling Dense Deterministic Answer Maps from a Versioned Culinary Embeddings Model. We test 27 frontier large language model endpoints on 534 substitution, pairing and constraining tasks for tasks that request a 3-ingredient portfolio from 8 candidates and score all 56 resulting portfolios. We conducted multiplicity-controlled paired tests on 101 of 351 model contrasts for this task-set. The largest point estimate on this task-set was achieved by Grok 4.6 at 65.1. The same rankings for this task-set were also achieved on several independently-compiled panels (using a variety of familiar metrics, task filters, etc.) and 3 public Epicure checkpoints. We present a 3-seed post-training study where LoRA SFT of a Qwen3-0.6B checkpoint on 270 optimal answers for Epicure to score on this task-set resulted in a 13.3 point gain on 84 anchor-disjoint maps (compared to format and label-matched control; 95% CI: 6.52, 20.29; p = 0.000170).

DOI arXiv Semantic Scholar

View source

Similar papers

FlavourBench : Ranking Frontier Language Models with Executable Culinary Ground Truth

FlavourBench, an automated benchmark in which a versioned culinary system supplies dense, executable ground truth, is introduced, an automated benchmark in which a versioned culinary system supplies dense, executable ground truth.

J. Chen, Erim Hayretci · 0 citations

#artificial intelligence Preprint Aug 2026

Test-Time Scaling in the Wild: Why Exploitation, Not Exploration, Is the Bottleneck

The first compute-normalised comparison of five TTS families across five open-ended generation benchmarks spanning medicine, law, finance, general chat, and creative writing is conducted - grounded in a unified framework that decomposes the effectiveness of each method's token budget into exploration and exploitation.

Davide Romano, Kanak Raj, Jerrod Parker et al. · 0 citations

Preprint Jul 2026

RepBench: Compiling Benchmarks into Capability Representations for Large Language Models

Under cross-benchmark transfer evaluation across twelve models completed by all four readouts, difference-in-means attains the highest model-level mean on ten models, while logistic regression wins the most capability-model cells.

Yanshi Li, Xue Bai, Shuman Liu et al. · 0 citations

May 2026

FindStatBench: Evaluating Large Language Models on Combinatorial Code Synthesis

Overall, statistic synthesis is much easier than map synthesis, some collections remain near-zero, long prompts cause a sharp accuracy cliff, and exact symbolic rule induction remains brittle.

Soham Dan · arXiv.org · 0 citations

Review Jul 2026

ConfidenceBench: Evaluating Confidence Calibration in Large Language Models

ConfidenceBench, a calibration benchmark that evaluates verbalized confidence estimates in 15 frontier LLMs using the Brier score, a proper scoring rule that incentivises truthful probability reporting shows that verbalized confidence calibration is a distinct and practically important axis of LLM reliability, compleme...

M. ffrench-Constant, Daniel Yang, Xinmeng Huang et al. · arXiv.org · 1 citation · ⚡1

Preprint Aug 2026

UpgradeBench: A Decision-Centric Benchmark for Upgrading Fine-Tuned LLM Specialists

This work presents UpgradeBench, a decision-driven longitudinal benchmark covering four consecutive Qwen releases, one continuation checkpoint, six tasks, and two model sizes, augmented by OLMo checkpoints with known training lineage, and disentangles three core questions: whether a new checkpoint improves fixed-recipe...

Yefei Chen, Wei-Ning Zhang · 0 citations