Author(s): Chew Loong Nian – AI ENGINEER

Originally published on Towards AI.

How an “agent skill” actually is

A DX engineer at WorkOS named Nick Nisi did something that sounds like sabotage. He took a 10,000-line library of auto-generated “skills” he had carefully built for his AI coding agent, deleted 95% of it down to 553 lines, and watched task accuracy climb from 77% to 97%. The eval suite that used to take 68 minutes finished in six. He didn’t add a smarter model. He didn’t tune a prompt. He removed almost everything and the agent got better.

The article explains that “skills” (e.g., Claude/Agent Skills folders with SKILL.md) work via progressive disclosure—metadata first, then full bodies only when relevant, and reference files only when needed—so more skills shouldn’t automatically help. It argues that adding lots of skills can make agents worse because of transformer limits: context tokens compete for attention, long contexts rot, information can get “lost in the middle,” and bloated tool/instruction ecosystems increase confusion and hesitation. Nick’s viral result came from deleting most lines after A/B evaluations that compared outcomes with a skill present vs absent, using deltas across many cases to identify harmful or dead-weight skills. The piece then distills practical guidance: keep skills short and gotcha-focused, write descriptions as triggers, A/B test and delete anything that doesn’t improve accuracy, shift multi-step orchestration into code instead of prose (instructions decay while code enforcement persists), and use audits and scripts to enforce conventions structurally. The closing takeaway is that agent quality increasingly comes from context curation and measurable eval deltas, not from stuffing more instructions.

Read the full blog for free on Medium.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI