We gave our AI agent 500 skills. It needed a graph, not a list.

We found out one of our AI skills had quietly become worse than the thing it was copied from, and nobody had noticed for weeks.

The skill was a fork of an open-source original called strategic-compact, pulled into our Claude Code plugin marketplace because we wanted to tune it. After we forked it, the upstream project kept moving: it gained a context-size signal with thresholds that scale to the model's context window. Our copy still ran the old logic, which only counted tool calls. Anyone invoking our version was getting a strictly worse skill than the one sitting in the upstream repo, free, one install away.

Here is the part that actually stung. When we audited the other skills we had forked from the same source, three of them differed from their upstreams only in frontmatter indentation. We had taken on the full maintenance burden of four forks and three of them added literally nothing. A fork's staleness is invisible until someone diffs it, and nobody diffs anything until something breaks.

That incident became a rule in our repo's CLAUDE.md: close gaps, never duplicate. But a rule written in prose is a hope, not a mechanism. This is the story of how we turned it into a mechanism, and what else fell out when we did.

Flat lists fail silently

A Claude Code session with our stack loaded carries several hundred skill descriptions in context, presented as a flat list of names and trigger text. Our own marketplace contributes six plugins, 39 skills, and 43 commands to that pile.

At that scale we kept hitting the same class of bug wearing different costumes. Two versions of one plugin got enabled at the same time, so every overlapping hook fired twice on every matching tool call. A skill existed at both user level and plugin level, and the two copies competed for the same invocation. Three different routing mechanisms grew up independently (a bash hook that grepped prompts for keywords, a skill-finding skill, and a routing skill), each answering "which capability applies here" from its own hand-maintained source.

Notice what these failures have in common. None of them is about a skill being bad at its job. Every one is about how skills relate to each other: this one duplicates that one, this one is a fork of that one, this one replaced that one, these two fire on the same trigger. A flat namespace has no way to say any of that. The relationships lived in our heads, in prose files, and in commit messages, which is to say they drifted.

Graph databases are having a moment for exactly this reason. Component registries and code maps already model software as nodes and edges because "what depends on what" is the question that matters. We decided the skill library deserved the same treatment.

What the graph actually is

Nothing exotic. A build script walks the repo and emits one JSON file: skills, plugins, commands, hooks, tags, MCP servers, and external upstreams as nodes, and a dozen typed edges carrying the relationships we kept failing to track: contains, fork-of, supersedes, overlaps-with, depends-on, replaces, extends, precedes, follows, augments, remediates, and usage-cooccurs. Some are structural, some are declared intent, some are mined from usage logs, and every edge records which of those it is. The newer types earn their keep at specific moments, which we'll get to.

Two design choices did most of the work.

Every edge records where it came from. A fork-of edge knows it was parsed from the provenance ledger; a tag edge knows it came from skill frontmatter; a usage edge knows it came from the monitor. When an edge is wrong, you know which source to fix.

The tag vocabulary is closed. Skills declare topics and stacks in frontmatter, but the compiler validates every slug against a curated catalog of 25 topics and 9 stacks, and an unknown slug fails the build. We added that after realizing the alternative: a typo'd tag would silently mint a real node in the graph and a real routing signal, indistinguishable from an intended one. In a generated artifact, a typo is not a cosmetic problem. It is data corruption.

The build is deterministic, byte for byte. Run it twice, get identical files. That let us add a staleness check: our weekly audit rebuilds the map, and if the committed artifact doesn't match the sources, the audit commits the fix and stops. A stale map is the fork problem all over again, so the map is not allowed to be stale.

Generated or nothing

The strongest argument for "generated, never hand-maintained" came from our own code, the same day we wrote it.

During the cleanup review of this very build, a reviewer found that our test fixture validator was a verbatim copy of the real validator, about 85 lines duplicated into a second file. By the time the review ran, the two copies had already diverged: the real one had gained a duplicate-id check the fixture copy lacked. Our fixtures were being validated by a weaker validator than the one guarding CI.

Read that timeline again. We were mid-project, actively building a system whose entire thesis is "copies drift and nobody notices," and our own copy drifted before the day was out. Not weeks. Hours. If you take one thing from this article, take that: the drift rate of duplicated logic is much faster than anyone's intuition, including ours, and we were primed to look for it.

So nothing in the map is hand-maintained. The one curated input is the tag catalog, and even that is enforced by the compiler rather than trusted. The docs that describe the skill inventory are themselves rendered from the map into marked sections of each README, and the render script refuses to run if the markers are missing rather than guessing where to write. Regeneration is idempotent. The inventory tables in our documentation can no longer disagree with reality, because they are no longer written by anyone.

What it unlocked day to day

The first consumer was routing. Our old prompt-time skill suggester was a bash script full of hand-written keyword patterns. It could not see new plugins, encoded zero relationships, and needed editing every time the library changed. The replacement reads the graph, matches the prompt against tag and name signals, and suggests at most one skill. It needs two independent matching signals before it will speak at all, and it fails silent on any error, because a wrong suggestion on every prompt is worse than none. Warm cost is about 50 milliseconds.

The second consumer was session startup. Four of our plugins used to print unconditional banners into every session: here are my commands, here is my context. Useful the first time, pure context tax the thousandth. They are gone, replaced by one hook that fingerprints the repo you are actually in (a Next.js config here, an Obsidian vault there) and surfaces up to eight skills relevant to that stack. Extends edges make the disclosure progressive: when a base skill and its specializations all match, the hook surfaces the base with a note that deeper skills exist, rather than spending its whole budget on one family. On repos where nothing matches, it prints nothing. Running it on the marketplace repo itself produces silence, which is correct, and honestly the silence took discipline to ship. Every instinct says print something.

The right skill at the right moment

Once routing and disclosure worked, a pattern became visible: a skill library doesn't have one moment where a suggestion helps, it has four. Session start (what's relevant to this repo). Prompt time (what's relevant to this request). After a skill finishes (what usually comes next). And when something breaks (what fixes this). Each moment wants a different relationship, and trying to serve all four from topic tags alone is how routers get noisy.

The subtlest addition was augments, for skills that improve a whole category of other skills rather than any one of them. Our humanizer skill makes any content-writing output sound less machine-generated. That's not a specialization of a content skill, and listing pairwise edges to every content skill would drift the moment a new one lands. So augments targets the category itself: one edge to the content-authoring tag covers every current and future member, including skills in plugins we don't own. When the router surfaces a content skill, it can mention the modifier that makes the output better.

For the something-broke moment, we added a third kind of tag: conditions. A condition like build-failure or type-error is a small curated entry that carries detection patterns, regexes matched against failing tool output rather than against anything the user typed. Skills declare which conditions they remediate, and a hook watches failed commands and suggests the top remediator. The satisfying part is who gets suggested: the best build-fixers in our ecosystem are third-party agents from the ecc plugin. The same map that guards against duplicating third-party capability now routes to it when it's the right tool. That's what complementary actually means.

And for the moment after a skill finishes, succession: a declared precedes edge for real pipelines (our PRD-writing command is followed by the PRD-grilling command, by design), backed by a mined follows edge derived from session logs when two skills keep occurring in order across sessions. Run the first step of a workflow and the hook offers the second. All four surfaces share one discipline, because it earned its keep in the router: at most one suggestion, silence otherwise, and no graph traversal at runtime.

The graph said no

The moment that convinced us this was more than plumbing came mid-build, from one of the agents doing the work.

Our implementation plan said: six hook scripts were left stranded in the wrong plugin by an old migration, move them to their proper home. The agent assigned to that task did something better than follow instructions. It diffed the six scripts against the target plugin first, and found that four of them already had live, wired, documented counterparts there. Two of those counterparts were byte-identical to the "stranded" copies. Two were newer. The plan's premise was wrong: those four were not stragglers waiting to be migrated, they were stale duplicates of a migration that had already happened. Moving them would have created a third copy of each.

The agent stopped, laid out the evidence, and proposed retiring the duplicates instead. That is the curation rule being enforced by relationship data rather than by vigilance. It is also, not coincidentally, exactly the failure mode the graph's fork-of and replaces edges exist to model. We shipped the retirement.

The same rule now runs at the front door. Our skill-ingest tool, Skill Forge, takes the map as an input and checks a candidate skill's name and description against every skill already in the graph before promotion. Strong overlap holds the candidate for review, with one deliberate exception: a candidate that declares it extends an existing skill is claiming layering, not duplication, and the gate turns that one pair into a reviewer's note instead of a block. Without the extends edge, the gate we built to stop duplicates would eventually punish exactly the deepening we want. "Does an enabled plugin already ship this?" used to be a question someone remembered to ask during review. Now it is a query.

One boundary we drew in the drift checker deserves a mention because it is easy to get wrong. The provenance ledger that records where each fork came from also contains prose describing how to check each fork for drift. The checker treats every bit of that as data. It reads files, fetches upstream content, hashes bytes, and compares. It never executes anything the ledger says, and a test pins that invariant. If you build anything that walks a graph of third-party references, make that rule structural, not behavioral.

Different is not the same as unreviewed

The drift checker's first real run embarrassed us. We pointed all seven forks at their upstream repos and every single one came back "drifted." Technically true. Practically useless. The only difference, in every case, was the five-line block of tags we had deliberately added — our own improvement, sitting in version control, flagged as if the upstream had moved underneath us. A two-way comparison, our file now against their file now, cannot tell those two situations apart. And a check that cries wolf seven times a week is a check people stop reading.

The fix was to give the checker a memory. When we ingest or review a fork, we record a baseline: the hash of the upstream as of the last time a human actually looked at it, stored as plain data in the same provenance ledger that records where the fork came from. The build also computes a normalized hash of our local copy with the injected tag block stripped out, so the one edit we made on purpose is invisible to the comparison.

Two comparisons instead of one yields four verdicts instead of two. Local matches baseline, upstream matches baseline: in sync, say nothing. We changed and upstream didn't: local-only — that is our work, it is in git, stay quiet. Upstream changed and we didn't: upstream-moved — the exact situation the whole system exists to catch, queue it for review. Both changed: diverged, queue that too. Only the last two page a human.

Re-baselining is the deliberate human act in the loop. Review what upstream did, adopt it or decline it, then run one script that re-records the baseline hash — the machine's way of writing down "seen it." After this shipped, the same seven forks report in sync, and the weekly audit's drift section went from seven permanent false alarms to zero. The check only speaks when it has something to say, which is the only condition under which anyone keeps listening.

The usage layer we didn't have to instrument

We assumed usage-weighted routing would require new telemetry. It didn't. Our existing skill monitor already logs which skills fire, keyed by session, and it turns out subagent transcripts share their parent's session id. That means skill co-occurrence across an entire delegated session was already sitting in the logs, needing only aggregation. Counts only, no prompt text. First build: 191 candidate skill pairs, of which 2 survived as real co-occurrence edges. Small numbers, honestly, but they are real, and they accumulate weekly.

The more useful metric turned out to be a structural one we named routing misses: skills that get used but that the router could never have suggested, because they have no tag edges to match on. That list feeds our refinement queue with a new kind of fix. Sometimes the right change is not to the skill at all. It is to the map.

Then we measured it

Everything above is argued from incidents — persuasive, but anecdotes. So the last thing we built was an eval suite, and the ground truth came from the map's own philosophy: mine it, don't author it. Every prompt in our session history that was followed by a real skill invocation is a labeled routing example. We restricted the mining to sessions from before the router went live, so it couldn't be graded on ground truth it helped create, and we resurrected the retired grep suggester from git history to serve as the baseline.

The old suggester scored zero. Not low — zero. Across every positive example in the golden set, the hand-written keyword patterns never once named the skill that actually got used. The map router hit 57 percent top-1. On the metric we care about more — silence precision, staying quiet on the large majority of prompts that deserve no suggestion — the router scored 76 percent against the old script's 20. Session-start disclosure now injects about 490 bytes in a repo where something matches and nothing anywhere else, versus the 5,187 bytes of fixed banners it replaced. Drift false alarms: seven before the baseline system, zero after. Small print, because honesty is the whole point of measuring: the first golden set holds just seven positive examples. It grows every week as history accrues.

The best result wasn't a number. Writing the curation-gate regression test — the five historical duplicates that started this whole project must be flagged, forever — meant cutting test fixtures from the real generated artifact instead of writing them by hand. The first run against real data found the gate had been reading a field no real artifact sets. It had been silently matching nothing since the day it shipped, and every hand-built test fixture passed anyway, because hand-built fixtures encode the author's assumptions, not the artifact's reality. That's the same lesson this article keeps paying for: the copy you maintain by hand lies to you. The gate is fixed, and all five duplicates now get caught — including the four that required checking against the third-party ecosystem inventory, not just our own catalog.

And the measuring runs without us. Every suggestion any hook fires is logged — skill ids and truncated hashes, never prompt text — and the weekly audit now appends one metrics line per run: drift verdict counts, the routing headline numbers, and per-hook acceptance rates from that log. A suggestion that keeps getting ignored is a routing miss, and a routing miss is a map bug, not a skill bug. When any of these numbers regresses, it surfaces in a week, on a schedule, instead of when someone remembers to look.

What we didn't build

Still no graph database. The map is a JSON file, and Neo4j or a temporal graph like Graphiti earns its complexity when the queries outgrow the file, not before. What we built instead is Cypher-shaped without Cypher, in two tiers. The hooks on hot paths never query at all: the build materializes precomputed indexes (token to skill, stack to skills, condition to fixers, skill to successor) deterministically, covered by the same staleness check as the map itself. Everything else, the weekly audit included, uses named queries defined as data: what-remediates, what-extends, overlap-candidates, unroutable-skills. If the map ever graduates to a real graph engine, those named queries port mechanically. No embedding-based routing either; lexical matching against a closed vocabulary is inspectable, and we want to exhaust it first. And there is one metric we documented instead of faking: "suggested but ignored" needs a log of what the router suggested, which does not exist yet. The honest version of a metrics section says which numbers you cannot compute.

If you run an agent skill library

Past a few dozen skills, your failures stop being about individual skills and start being about relationships between them. Model the relationships explicitly, in a generated artifact with provenance on every edge, and make staleness a build failure rather than a discovery. Close your tag vocabulary, because in generated data a typo is corruption. Prefer silence to noise everywhere your tooling talks to the model. And if part of your library is forked from somewhere, either diff it on a schedule or delete it, because the copy is already drifting, and it drifts faster than you think. Ours didn't last the day.