Erdni Zergetaev

Measuring the Wrong Number: Cost-Aware, Correctness-Gated Evaluation of Efficiency Claims in Multi-turn Agentic Coding

A fast-growing ecosystem of plugins, skills, and prompt packs promises to make AI coding agents cheaper. The most popular single artifact—a "talk-like-a-caveman" skill for a widely used coding agent—has ~92,000 GitHub stars (as of 2026-07) and advertises a 65% token reduction. We show that such claims, though not fabri...