Skip to content
#software testing Open access

Dual Mesh Benchmark DeepSeek Column: 20/20 Conforming Resolutions, 8.5 s Median, by a Previously Non-Conforming Engine After Preregistered Repair

Oct 2026 · Zenodo (CERN European Organization for Nuclear Research) · 3 references
Topic Modeling

Abstract

Preregistered benchmark leg, not a certification. DeepSeek v4-pro (via OpenRouter, model_returned verified per call) resolved the first 20 instances of the frozen CORPUS_B1.json synthetic anomaly corpus (sha256 13295d19...), the identical selection rule as every prior B1 baseline leg, one stateless request per instance, wall-clocked request-to-response. Results: 20/20 strict-valid conforming tuples (zero normalization needed), 20/20 correct against expected playbooks including exact platform playbook_ids, median 8,538.0 ms, p95 11,864.0 ms, total API cost $0.0089. Comparisons are directional only (XPLAT-2/3 doctrine: raw external API is not the conversation-loop substrate): the v1.0 mesh lookup median on the same corpus is 0.000443 ms; the v1.0 Automatic conversation-loop baseline median is 4,509.1 ms. The raw-API DeepSeek median exceeds the full-loop figure, attributable to reasoning-token generation rather than platform orchestration. Accuracy-per-dollar is excellent; latency under reasoning defaults is not this engine's lane. No wall-clock equality claim across substrates is made. Honest-results note: this column completes a documented repair. The Oct 11 CCR-4 DeepSeek capture was NON-CONFORMING (fabricated tuple, wrong array shape) and remains on the public tally. The preregistered containment re-run (CCR-4X leg I: conforming 9/9, forged APPR-9923 authority rejected) and this benchmark leg followed only after that loss was posted publicly (publish-first repair doctrine). Limitations: sampled baseline (n=20), single engine and route, October 2026; synthetic corpus only; providers varied by call (StreamLake, Parasail) per OpenRouter routing; no stress/load or live-healing testing; not a model certification or universal performance claim. Dual Mesh Benchmark: this record is itself a scoped DMB measurement; all other DMB claims remain NOT MEASURED here. Bundle contains the receipt (Markdown), machine-readable receipt JSON, full evidence archive (prereg, runner, results, all 20 raw responses and extracted contents, SHA-256 manifest). Registration and evidence: public four-horsemen ledger (prereg commit aa751dd before execution; receipt commit 9bc6851). Research home: DCAI; portal: DSOS Science. Lineage acknowledged: Nyquist, Hartley, Shannon, Kolmogorov. No new mechanism is claimed. This software is a prototype and is provided for educational and research purposes only. It is not intended for production use, commercial deployment, or safety-critical environments. All systems are experimental and may contain defects on them. Version note (v2): this version supersedes v1 solely to redact verbatim operator chat text from the preregistration per the operator-privacy directive (Oct 13, 2026). Authorization is recorded as GO + timestamp only. No measurement, result, or claim changed.

View source

Similar papers

#computer vision Review Sep 2017

Agile Software Development Methods: Review and Analysis

This publication proposes a definition and a classification of agile software development approaches and analyses ten software development methods that can be characterized as being "agile" against the defined criterion.

P. Abrahamsson, O. Salo, Jussi Ronkainen et al. · 727 citations · ⚡54
#computer vision Jun 2008

The impact of agile practices on communication in software development

The study shows that agile practices improve both informal and formal communication, but indicates that, in larger development situations involving multiple external stakeholders, a mismatch of adequate communication mechanisms can sometimes even hinder the communication.

M. Pikkarainen, Jukka Haikara, O. Salo et al. · 401 citations · ⚡48
#machine learning Review Open access Oct 2014

Software development in startup companies: A systematic mapping study

The results indicate that software engineering work practices are chosen opportunistically, adapted and configured to provide value under the constrains imposed by the startup context.

Nicolò Paternoster, Carmine Giardino, M. Unterkalmsteiner et al. · 394 citations · ⚡54
#computer vision Review Mar 2008

Agile methods in European embedded software development organisations: a survey on the actual use and usefulness of Extreme Programming and Scrum

The results show that the embedded industry has been able to apply agile methods in its development processes and that the appreciation of the agile methods and their individual practices appears to increase once adopted and applied in practice.

O. Salo, P. Abrahamsson · 238 citations · ⚡9
#computer vision Open access Jul 2017

What happens when software developers are (un)happy

Consequences of happiness and unhappiness that are beneficial and detrimental for developers' mental well-being, the software development process, and the produced artifacts are found.

D. Graziotin, Fabian Fagerholm, Xiaofeng Wang et al. · 236 citations · ⚡13
#computer vision Open access Oct 2004

Mobile-D: an agile approach for mobile application development

The Mobile-D approach is briefly outlined here and the experiences gained from four case studies are discussed, which helped develop an agile development approach for mobile application development.

P. Abrahamsson, Antti Hanhineva, H. Hulkko et al. · 225 citations · ⚡18

Related blog posts

Microsoft Research Blog Oct 12, 2026

Simulating student learning for better AI tutoring

StudentSim is a research system for evaluating and improving AI tutoring tools before they are tested with learners. It models how individuals may engage with tutors, giving feedback to improve tutor performance across subjects like chess, writing, and mathematics. The post Simulating student learning for better AI tutoring appeared first on Microsoft Research.

MIT News · Artificial Intelligence Oct 2, 2026

Documenting the tech worker movement

Writing as a participant and researcher, PhD student JS Tan SM ’22 has co-authored a new book about the rise of tech worker protests and the employer backlash that followed.

MIT News · Artificial Intelligence Oct 2, 2026

Documenting the tech worker movement

Writing as a participant and researcher, PhD student JS Tan SM ’22 has co-authored a new book about the rise of tech worker protests and the employer backlash that followed.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.