Skip to content
Preprint

Praxist: From Experimental Artifacts to Solution Lineages

Aug 2026 · 0 citations
Computer Science

TL;DR

Praxist is introduced, a lineage-centered generational system that converts reproducible artifacts and evaluator outcomes into a typed evidence graph of findings, lane-structured frontiers, and agendas, and Separating local artifact construction from cohort-level evidence synthesis lets later attempts inherit validated mechanisms, unresolved claims, and useful constraints.

Abstract

Autonomous R\&D agents now write, run, and improve executable artifacts under automated evaluation---but largely as laboratory instruments: shown on curated benchmarks, with gains that are hard to trace to a cause and costs well above what sustained engineering practice absorbs. The limitation is structural. Most systems treat each attempt as nearly self-contained, so logs, memories, and search trees record what happened without establishing which design element produced an improvement, whether its evidence survived validation, or how it recombines with others. Long campaigns therefore keep re-learning the same lessons. We introduce Praxist, a lineage-centered generational system that converts reproducible artifacts and evaluator outcomes into a typed evidence graph of findings, lane-structured frontiers, and agendas. Separating local artifact construction from cohort-level evidence synthesis lets later attempts inherit validated mechanisms, unresolved claims, and useful constraints, and leaves results attached to an inspectable lineage. On the standardized 75-task MLE-bench suite, the finalized official-grader results give Praxist 60 medals (80.0\%), 49 of them gold, against 55 medals (73.3\%) and 34 gold for a Claude Code baseline on Claude Opus 4.8---at a recorded model spend of US\$3,054 versus US\$38,370, roughly a twelfth of the cost. Four case studies---quantitative trading, LiDAR-inertial-visual SLAM, tokamak magnetic control, and rocket landing---carry the same process into open-ended engineering problems, improving on each task-native baseline in headline accuracy, survival, or resource cost, with the discovery path on record. Stronger artifacts at an order of magnitude less spend, each backed by an auditable lineage, are, to our knowledge, first brought together here: the operating profile production research requires, not the one a benchmark demonstration establishes.

View source

Similar papers

Preprint Aug 2026

One Recipe, Many Harnesses: What Self-Evolution Encodes Across Languages and Models

Together, these results recast the evolved harness as a legible compensation layer, shaped jointly by the language's engineering demands and the model's behavioral gaps, rather than an opaque benchmark-tuned scaffold.

Siqi Yang, Qianlan Yang, Yu-Xiong Wang et al. · 2 citations
#artificial intelligence Preprint Sep 2026

Bioinfoysis Technical Report

A multi-agent harness that represents each request as a persistent, artifact-grounded analysis run, and demonstrates that reliable bioinformatics automation depends not only on model capability, but also on the harness that governs planning, execution, memory, and evidence flow.

Qi Shao, Xin Zhang, Zhou-Yang Yuan et al. · 0 citations
Preprint Aug 2026

Beyond Execution: Auditing Experimental Fidelity in LLM-Driven Scientific Research

ABE-Ralph is introduced, a reference-anchored auditing framework that represents claims, protocols, required components, baselines, and metrics as structured experimental constraints, guides implementation through an 8-step workflow, and performs quantitative, qualitative, and code-level verification.

Le-Zhi Yu, Xiao-Gang Xu, Yuhong Zhou et al. · 0 citations
Jul 2026

AgentHPOBench: A Benchmark For Evaluating LLM Agents as Sequential Hyperparameter Optimizers

AgentHPOBench, a sequential benchmark comprising 30 executable machine learning tasks across seven research categories, shows that current agents exhibit measurable experimental optimization ability across domains, but still face clear limitations in sustained iterative refinement, complex log diagnosis, and consistent...

Tianyu Huai, Tingshuo Fan, Xinchi Chen et al. · 0 citations

ORCA-bench: How Ready Are Language Model Agents for Oncall?

The ORCA-bench benchmark is introduced, a benchmark that puts general-purpose coding agents in a production-fidelity oncall setting and is a lower bound on the engineering investment required before frontier coding agents can be safely entrusted with production reliability.

Albert Gong, Kyuseong Choi, Abhineet Agarwal et al. · 1 citation
Review Aug 2026

Agentic Artifact Creation: Systems, Evaluation, Principles, and Opportunities

This survey examines agentic artifact creation, which is defined as stateful construction in which an AI system materially constructs or revises a deliverable and intermediate observations redirect later work, and formulate principles for keeping commitments and responsibility explicit, turning feedback into targeted r...

Tianfu Wang, Zhe-Zheng Hao, Xinchi Xia et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.