This note argues that a skill is a software artefact and that its construction should follow software-engineering principles, with qualifications: single responsibility, separation of interface from implementation, low coupling, and economy in a shared token budget, together with behavioural evaluation in place of deterministic testing.
Abstract
Agent Skills are an emerging way to extend large language model agents with reusable procedural knowledge that the agent loads on demand. Anthropic introduced Agent Skills and published the format as an open specification supported across several agent tools. This note argues that a skill is a software artefact and that its construction should follow software-engineering principles, with qualifications: single responsibility, separation of interface from implementation, low coupling, and economy in a shared token budget, together with behavioural evaluation in place of deterministic testing. Using Claude Code as the reference implementation, it describes how a skill is structured, how its contents are loaded in stages, and how to write the description on which selection depends. It places skills against the other mechanisms a developer can use to shape agent behaviour, like project memory files, slash commands, subagents, external tool connections, and hooks, and gives a rule for choosing between them based on who decides that a mechanism runs and what guarantee it provides. It then sets out an evaluation-driven authoring process, a set of patterns and faults commonly encountered in authoring, and the trust question raised by using skills from third parties. We illustrate the comparison drawn in UML class style, the loading model, the anatomy of a skill, the relative position of each mechanism, and the points at which skills and hooks act during a session.
Skill Compilation is introduced, realized in SIGIL, which compiles a prose skill into an executable harness, and is model-independent: the harness holds at 86% across two model generations while prose swings from 56% to 68%.
Jayanaka L. Dantanarayana, Savini Kashmira, Lingjia Tang et al.· 0 citations
Agent skills provide a reusable way to specify multi-step agent behavior, but they remain natural-language specifications interpreted by the model at runtime. As a result, required tool calls, ordering constraints, and checks may be skipped even when explicitly prescribed by the skill. We introduce Skill Compilation, a paradigm that translates natural-language skills into executable agent programs while preserving model judgment where semantic decisions are required. We realize this idea in SIGIL. SIGIL extracts source-grounded requirements, decomposes them using a closed Agent Instruction Set (AIS), composes them into AG-IR with explicit ownership, data flow, and control flow, and deterministically lowers validated AG-IR into executable code. Across 33 publicly available SKILL.md files and three runtime models, SIGIL increases mean Applicable-Mandate Compliance (AMC), the fraction of applicable skill requirements satisfied during execution, from 66.0% with direct skill execution to 88.6%. SIGIL also reduces total runtime token consumption by 2.40-5.95x. These results show that compiling procedural structure improves the reliability and efficiency of skill execution while retaining model judgment where it is needed.
Jayanaka L. Dantanarayana, Savini Kashmira, Lingjia Tang et al.· 0 citations
Enterprise agent programs are moving from prototypes into production, where reusable skills, tools, and workflow packages must be reviewed with evidence rather than prose. Current gates often scan these artifacts for structure, style, and security, but they do not answer the deployment question: does the capability package help a live agent complete enterprise tasks under the same model, sandbox, and grading policy? We present ACES (Agentic Continuous Evaluation of Skills), a repository-native framework for evaluating skills and product capability packages as executable agent artifacts. ACES runs paired live trials with and without a target skill, normalizes trajectories into the Agent Trajectory Interchange Format (ATIF), grades six default runtime metrics, and reports Skill Lift: the target skill's added value for a fixed task, harness, workspace, and scorer. The same protocol supports product-owned task suites that compare baseline, skill, bundle, team-skill, and plugin targets. On 145 real skills from internal enterprise repositories and public catalogs, scan-only gates surface useful authoring issues but measure complementary facets (structural versus LLM-judge Spearman $\rho = 0.14$). Across 947 scored paired cases from 58 of 64 production skills and four primary harnesses, mean composite Skill Lift is 0.2134 (95\% paired-case CI [0.1967, 0.2301]); mean outcome-only lift, the average of accuracy and goal accuracy, is 0.1799. Composite lift is positive in 72.8\% of paired cases. The largest process-metric gains appear in skill execution, behavior check, and skill efficiency---signals about discovery, routing, workflow following, and tool use that document scans cannot observe. An open-source implementation of the methodology is available in NVIDIA SkillEvaluator.
Christopher Kevin, Narendran Raghavan, J. Puget et al.· 0 citations
IDE Agent Mode is changing how the developers work, from writing code to debugging & software management, by embedding powerful AI features right inside popular development setups. This article first conceptualizes IDE Agent Mode as a workflow where AIs are coding partners that not only understand the context of the project but are also able to write code, suggest better solutions, automate routine tasks, and even help with debugging at the same time. With the increase in the complexities of software projects and the trend towards shorter development cycles, AI-driven software development has become an effective way to enhance developers' productivity, minimize human errors, and foster innovation. On the other hand, deciding which AI model should be used for which programming task remains one thorny issue that developers and companies face. This is due to significant differences among models in their performance, speed, reasoning capabilities, cost, management of context, and compatibility with the development tools. This article not only lists the AI models that are most often used in IDE Agent environments but also assesses their performance in various coding assistance tasks such as code completion, bug fixing, documentation generation, and architectural reasoning. The method includes a comparative study, obtaining developer feedback, and assessment of the practical workflow in order to determine the pros and cons of different models in real-life software engineering situations. Results show that there is no 'silver bullet' model that fits all requirements; rather, the choice of model should be tailored to the project's needs, the professional level of the team, and scalability requirements, as well as financial constraints. The research also reveals that small models serve very well for fast coding assistance, whereas the more feature-heavy models are quite apt at doing complex tasks such as debugging and system-level designing.
Madhurima Kommuru· International Journal of Mod...· 0 citations
MAGE explains how externalized knowledge, bounded action, independent evaluation, and retained human authority can compose into a governed engineering environment, and proposes tests of when that environment turns commodity intelligence into durable engineering progress.
James C. Davis, Kelechi G. Kalu, Huiyun Peng et al.· 1 citation
Under the current standard, Agent Skills are SKILL.md files that combine instructions with supporting resources, enabling Large Language Model (LLM) agents to reuse procedures beyond a single conversation. Yet many public skills appear to originate from a single task, repository, or conversation, even when they are shared as reusable components. We analyze this gap across 138,133 public SKILL.md files from 20,556 repositories using a two-tier defect taxonomy grounded in the official specification and best-practice guidance. We find that 91.8% of skills contain at least one detected defect, with stable estimates across lenient and strict thresholds (88.8-94.6%). The dominant failures are ordinary packaging problems rather than exotic attacks: weak routing metadata, bloated or non-actionable bodies, and poor resource organization. A deterministic routing stress test over 20,000 skills shows the functional impact: skills with valid routing metadata are retrieved more reliably from startup descriptions than skills with routing defects. Defect rates vary by platform and provenance: specification-aware skills contain fewer defects, while AI-marked skills show more safety and portability problems. Lightweight enforcement and repair experiments support a quality-assured generation workflow combining spec-aware prompting, lightweight linting, automated repair, and safety gating.
Chi Zhang, Yimin Liu, Xinze Chen et al.· 0 citations