Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions app/components/OgLines.vue
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
<script setup lang="ts">
import type { OgFace } from '../utils/og-style'
import { computed } from 'vue'
import { OG_MUTED, ogEvenLines, ogLineStyle, ogTitleStyle } from '../utils/og-style'

// Centred OG text of up to `lines` lines, broken evenly when it needs two.
Expand Down
58 changes: 58 additions & 0 deletions docs/editorial/skill-md-size-study-2026-10-09.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Skill size study evidence

The study answers how Skill descriptions, bodies, and reference inventories differ across a frozen registry sample.
Harlan requested original research using the Unhead research articles as the editorial reference.

## Reproduction

The source measurements are committed under `public/research/skill-md-size-2026-10-09/`.
Run `pnpm exec tsx scripts/research/build-size-study.ts` to rebuild `summary.json`.
The provenance manifest records the original export hash, measurement hash, tokenizer, and rubric version.

Migration 0151 stores the same summary and provenance in D1.
The existing source-version description and body observations remain in `skill_generated`.
The research snapshot has no foreign key to live Skills. Source deletion must not erase published evidence.
Publish corrections with an explicit snapshot revision. Do not silently replace historical measurements.

## Claim checks

| Claim | Evidence |
| --- | --- |
| 12,141 filtered Skills, 2,297 repositories | Included rows and distinct repository identities |
| 12,138 matched bodies, three missing | Nonnull body token measurements |
| 8.9% reach 5,000 tokens | 1,078 / 12,138 |
| Repository weighting raises that to 10.8% | Mean of per-repository proportions, 2,296 measured repositories |
| Mean description rises from 345 to 383 characters | Skill mean versus mean of repository means |
| 11,760 normalised descriptions | Trim and collapse whitespace before SHA-256 |
| 11,764 exact descriptions | Hash exact description text before normalisation |
| File duplicates leave the tail near 8.9% | 1,061 / 11,983 measured unique raw files |
| Reference mean is 2.27, median zero | 12,091 complete inventories, nulls excluded |

Exact text uniqueness and whitespace-normalised uniqueness use different definitions.
Neither means the source bodies are distinct.
Body counts exclude frontmatter. Whole-file kB includes frontmatter.
All percentiles use nearest rank.

## Interpretation limits

The corpus is neither a random sample nor a top-installed leaderboard sample.
Existing aggregator flags predate the study. Their exclusion does not establish independent mirror detection.
Models see different tokens. o200k_base estimates cannot reproduce every Agent's catalogue truncation.
Jev scores have no independent human calibration. The article labels the semantic section exploratory.
No actual Agent selection, task completion, or corpus-wide skills-ref validation claim appears.
The public row dataset omits source instructions and descriptions. Pinned commits and paths identify those sources.
Word frequencies require the source descriptions, the published stop list, and the stated token rule to reproduce.

## Search and admission

NuxtSEO Pro queries on 9 October 2026 measured the US `skill md` seed at 720 searches/month, KD 28.
Specific size and description-length seeds returned no provider data. This is not measured zero demand.
The owner explicitly approved original research as an authority surface, independent of keyword volume.
The admission record names that purpose and the 11 November gate in `page-admissions.ts`.
Review relevant impressions, citations, dataset use, and authoring-guide clicks separately.

## Primary guidance

- [Agent Skills specification](https://agentskills.io/specification): description cap, progressive disclosure, body guidance, and format validation.
- [Claude authoring guidance](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/best-practices): concise instructions and real-usage tests.
- `scripts/lib/description-review.ts`: exact Jev questions, model version, and thresholds.
3 changes: 2 additions & 1 deletion eslint.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@ import harlanzw from 'eslint-plugin-harlanzw'

export default antfu({
// skilld sync writes .skills from skilld-dev/skilld. CI checks it with check:skills.
ignores: ['.skills/**'],
// Immutable research exports retain their recorded byte hashes.
ignores: ['.skills/**', 'public/research/**/measurements.json'],
rules: {
'vue/no-useless-v-bind': 'off',
'vue/attribute-hyphenation': 'off',
Expand Down
201 changes: 201 additions & 0 deletions layers/marketing/app/components/content/SkillSizeStudy.vue
Original file line number Diff line number Diff line change
@@ -0,0 +1,201 @@
<script setup lang="ts">
import summary from '../../../../../public/research/skill-md-size-2026-10-09/summary.json'
import vocabulary from '../../../../../public/research/skill-md-size-2026-10-09/words.json'

const { chart } = defineProps<{ chart: 'body' | 'topics' | 'descriptions' | 'words' }>()
const number = (value: number) => value.toLocaleString('en-US', { maximumFractionDigits: 0 })
const decimal = (value: number | null, digits = 1) => value === null ? 'Unavailable' : value.toFixed(digits)
const topicLabels: Record<string, string> = {
coding: 'Coding',
writing: 'Writing',
design: 'Design',
operations: 'Operations',
data: 'Data',
planning: 'Planning',
security: 'Security',
research: 'Research',
other: 'Other tasks',
unclear: 'Task category uncertain',
}
const totals = summary.topics.reduce((total, topic) => ({
green: total.green + topic.bands.green,
amber: total.amber + topic.bands.amber,
red: total.red + topic.bands.red,
}), { green: 0, amber: 0, red: 0 })
const bodyBands = [
{ label: 'At most 2,500 tokens', count: totals.green, class: 'study-green' },
{ label: '2,501 to 4,999 tokens', count: totals.amber, class: 'study-amber' },
{ label: '5,000 tokens or more', count: totals.red, class: 'study-red' },
]
const bandLabel = (min: number, max: number | null) => max === null ? `${number(min)}+` : `${number(min)} to ${number(max)}`
</script>

<template>
<figure class="study-chart not-prose">
<template v-if="chart === 'body'">
<h3>Most Skill bodies fit below 5,000 tokens</h3>
<p class="study-note">
12,138 measured bodies. Three source versions unavailable.
</p>
<div v-for="band in bodyBands" :key="band.label" class="study-band">
<div class="study-row-label">
<span>{{ band.label }}</span>
<span class="study-number">{{ number(band.count) }} Β· {{ decimal(band.count / summary.included.bodyMeasured * 100) }}%</span>
</div>
<div class="study-track" aria-hidden="true">
<span :class="band.class" :style="{ width: `${band.count / summary.included.bodyMeasured * 100}%` }" />
</div>
</div>
<div class="study-axis">
<span>0%</span><span>100% of measured bodies</span>
</div>
<dl class="study-stats">
<div><dt>Median body</dt><dd>{{ number(summary.included.bodyMedian!) }} tokens</dd></div>
<div><dt>Mean body</dt><dd>{{ number(summary.included.bodyMean!) }} tokens</dd></div>
<div><dt>90th percentile</dt><dd>{{ number(summary.included.bodyP90!) }} tokens</dd></div>
</dl>
<figcaption>Estimated with o200k_base. Colours show context use, not quality. Green marks half the recommended budget.</figcaption>
</template>

<template v-else-if="chart === 'topics'">
<h3>Avg Skill.md Size</h3>
<p class="study-note">
Whole-file size, including frontmatter. Task categories come from description assessment.
</p>
<div class="study-table" role="region" aria-label="Skill size by task category" tabindex="0">
<table>
<caption class="sr-only">
Means by task category, with measured Skill counts and reference coverage
</caption>
<thead>
<tr>
<th scope="col">
Task category
</th><th scope="col">
Skills
</th><th scope="col">
Description chars
</th><th scope="col">
SKILL.md kB
</th><th scope="col">
Reference files
</th>
</tr>
</thead>
<tbody>
<tr v-for="topic in summary.topics" :key="topic.topic">
<th scope="row">
{{ topicLabels[topic.topic] }}
</th>
<td>{{ number(topic.skills) }}</td>
<td>{{ number(topic.descriptionMean!) }}</td>
<td>
<span>{{ decimal(topic.fileMeanKB) }}</span>
<span class="study-mini-track" aria-hidden="true"><span :style="{ width: `${topic.fileMeanKB! / 16 * 100}%` }" /></span>
</td>
<td>{{ decimal(topic.referenceMean, 2) }}<small>{{ number(topic.referenceMeasured) }} inventories</small></td>
</tr>
</tbody>
</table>
</div>
<figcaption>1 kB = 1,000 UTF-8 bytes. Bars start at zero, scale ends at 16 kB. Missing reference inventories stay missing. Zero counts remain in the mean.</figcaption>
</template>

<template v-else-if="chart === 'descriptions'">
<h3>Shorter descriptions do not always give clearer selection guidance</h3>
<p class="study-note">
Jev 1.13 assessment of task, activation, scope, and filler. Observed association, without an Agent selection test.
</p>
<div class="study-legend">
<span><i class="study-clear" />Clear guidance</span><span><i class="study-uncertain" />Uncertain</span><span><i class="study-red" />Needs work</span>
</div>
<div v-for="band in summary.descriptionBands" :key="band.min" class="study-band">
<div class="study-row-label">
<span>{{ bandLabel(band.min, band.max) }} characters</span><span class="study-number">{{ number(band.total) }} Skills</span>
</div>
<div class="study-track study-stack" aria-hidden="true">
<span class="study-clear" :style="{ width: `${band.clear / band.total * 100}%` }" />
<span class="study-uncertain" :style="{ width: `${band.uncertain / band.total * 100}%` }" />
<span class="study-red" :style="{ width: `${band.needsWork / band.total * 100}%` }" />
</div>
<p class="study-note">
{{ decimal(band.clear / band.total * 100) }}% clear Β· {{ number(band.uncertain) }} uncertain Β· {{ number(band.needsWork) }} need work
</p>
</div>
<figcaption>12,141 descriptions. Labels describe model judgements under a published rubric, not proven performance.</figcaption>
</template>

<template v-else>
<h3>The words Skill authors use</h3>
<div class="study-cloud" aria-hidden="true">
<span v-for="word in vocabulary.words.slice(0, 32)" :key="word.text" :style="{ fontSize: `${0.8 + word.count / vocabulary.words[0]!.count * 2.1}rem` }">{{ word.text }}</span>
</div>
<details>
<summary>Word frequencies and counting method</summary>
<p class="study-note">
{{ vocabulary.method }} Font size is a visual aid. Counts below give the precise frequency.
</p>
<div class="study-table">
<table>
<caption class="sr-only">
Top 32 words counted once per description
</caption><thead>
<tr>
<th scope="col">
Word
</th><th scope="col">
Descriptions containing it
</th>
</tr>
</thead><tbody>
<tr v-for="word in vocabulary.words.slice(0, 32)" :key="word.text">
<th scope="row">
{{ word.text }}
</th><td>{{ number(word.count) }}</td>
</tr>
</tbody>
</table>
</div>
</details>
<figcaption>English alphanumeric words, counted once per description. Common words removed. This view does not represent every language.</figcaption>
</template>
</figure>
</template>

<style scoped>
.study-chart { margin-block: 2.5rem; padding: clamp(1rem, 3vw, 2rem); border: 1px solid var(--ui-border); border-radius: var(--ui-radius); background: var(--ui-bg); color: var(--ui-text); }
.study-chart h3 { margin: 0; font-size: 1.35rem; font-weight: 600; line-height: 1.4; letter-spacing: -.03em; }
.study-note, .study-chart figcaption { margin-top: .75rem; color: var(--ui-text-muted); font-size: .875rem; line-height: 1.6; }
.study-band { margin-top: 1.5rem; }
.study-row-label { display: flex; flex-wrap: wrap; justify-content: space-between; gap: .5rem; font-size: .9rem; }
.study-number, .study-axis, .study-stats dd { font-family: var(--font-mono); font-variant-numeric: tabular-nums; }
.study-track { display: flex; height: 1.15rem; margin-top: .65rem; background: var(--ui-bg-accented); }
.study-track > span { display: block; height: 100%; }
.study-green { background: #15803d; }
.study-amber { background: #b45309; }
.study-red { background: #be123c; }
.study-clear { background: #57534e; }
.study-uncertain { background: #a8a29e; }
.study-axis { display: flex; justify-content: space-between; gap: 1rem; margin-top: .8rem; font-size: .7rem; color: var(--ui-text-muted); }
.study-stats { display: grid; grid-template-columns: repeat(3, minmax(0, 1fr)); gap: 1rem; margin-top: 2rem; }
.study-stats dt { color: var(--ui-text-muted); font-size: .8rem; }
.study-stats dd { margin-top: .4rem; font-size: 1rem; }
.study-table { overflow-x: auto; margin-top: 1.5rem; }
.study-table:focus-visible, summary:focus-visible { outline: 2px solid var(--ui-primary); outline-offset: 3px; }
table { width: 100%; border-collapse: collapse; text-align: left; font-size: .8rem; }
th, td { padding: .9rem .7rem; border-bottom: 1px solid var(--ui-border); vertical-align: top; }
th { font-weight: 500; }
thead th { font-family: var(--font-mono); color: var(--ui-text-muted); font-size: .7rem; }
td { font-family: var(--font-mono); font-variant-numeric: tabular-nums; }
td small { display: block; margin-top: .25rem; color: var(--ui-text-muted); font-size: .65rem; white-space: nowrap; }
.study-mini-track { display: block; height: .3rem; width: 5rem; margin-top: .4rem; background: var(--ui-bg-accented); }
.study-mini-track > span { display: block; height: 100%; background: var(--ui-text-muted); }
.study-legend { display: flex; flex-wrap: wrap; gap: 1rem; margin-top: 1.25rem; font-size: .8rem; }
.study-legend span { display: flex; align-items: center; gap: .4rem; }
.study-legend i { display: inline-block; width: .65rem; height: .65rem; }
.study-cloud { display: flex; flex-wrap: wrap; align-items: center; justify-content: center; gap: .35rem 1rem; padding-block: 2rem; line-height: 1.25; }
.study-cloud span { font-weight: 500; }
summary { min-height: 44px; padding-block: .7rem; cursor: pointer; font-size: .9rem; }
@media (max-width: 480px) { .study-stats { grid-template-columns: 1fr; gap: .75rem; } .study-stats div { display: flex; justify-content: space-between; gap: 1rem; } .study-stats dd { margin: 0; } }
@media (min-width: 1024px) { .study-chart { width: calc(100% + 12rem); max-width: none; margin-left: -6rem; } }
</style>
31 changes: 31 additions & 0 deletions layers/marketing/app/pages/learn/[...slug].vue
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,37 @@ useSeoMeta({
useHead({
link: [{ rel: 'canonical', href: canonicalUrl }],
})

if (slug.value === 'research/skill-md-size-study') {
defineOgImage('Page.takumi', {
title: data.value.title,
description: '12,141 Skills Β· 2,297 repositories Β· Original skilld research',
}, { alt: data.value.title })

useSchemaOrg([
defineArticle({
headline: data.value.title,
description: data.value.description,
datePublished: data.value.publishedAt,
dateModified: data.value.updatedAt,
author: { '@type': 'Organization', 'name': 'skilld', 'url': siteOrigin },
}),
{
'@type': 'Dataset',
'@id': `${canonicalUrl.value}#dataset`,
'name': 'Skill size and description measurements, 9 October 2026',
'description': 'Frozen registry measurements with aggregator exclusions, source hashes, description assessment, and weighting comparisons.',
'url': canonicalUrl.value,
'version': '2026-10-09-v1',
'creator': { '@type': 'Organization', 'name': 'skilld', 'url': siteOrigin },
'distribution': ['csv', 'json'].map(format => ({
'@type': 'DataDownload',
'encodingFormat': format === 'csv' ? 'text/csv' : 'application/json',
'contentUrl': `${siteOrigin}/research/skill-md-size-2026-10-09/measurements.${format}`,
})),
},
])
}
</script>

<template>
Expand Down
5 changes: 5 additions & 0 deletions layers/marketing/app/utils/page-admissions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ export interface PageAdmission {

/** Pages added on or after 2026-09-04 that the audit covers. */
export const FREEZE_AUDIT_PATHS: readonly string[] = [
'/learn/research/skill-md-size-study',
'/agents',
'/agents/claude-code',
'/agents/codex',
Expand Down Expand Up @@ -59,6 +60,10 @@ export const FREEZE_AUDIT_PATHS: readonly string[] = [
]

export const PAGE_ADMISSIONS: Readonly<Record<string, PageAdmission>> = {
'/learn/research/skill-md-size-study': {
targetQuery: 'SKILL.md size and description length',
admissionBar: 'Owner-approved original research, 2026-10-09. One frozen registry study with source hashes, downloadable measurements, methods, weighting comparisons, and an authoring-guide action. Specific size queries have no DataForSEO estimate; the broader skill md seed has 720 US searches a month, KD 28. Research authority and citations are explicit goals, independent of search volume. Review indexing, relevant impressions, citations, and authoring-guide clicks at the 2026-11-11 gate. Retain useful cited research; consolidate into the authoring guide if it has no distinct use.',
},
'/skills/demos': {
targetQuery: 'claude skills examples',
admissionBar: '720 searches a month, KD 32. NuxtSEO research keywords, 2026-10-06. Holds only demos a human approved by merging, and renders noindex below six. Covers each demo page, `/skills/demos/<owner>/<repo>/<name>`, which targets "<skill> skill example" (owner-approved 2026-10-07; volume not measured). Review at the 2026-11-11 gate.',
Expand Down
2 changes: 1 addition & 1 deletion layers/marketing/content.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ export default defineContentConfig({
type: 'page',
source: {
cwd: marketingContentRoot,
include: 'learn/*.md',
include: 'learn/**/*.md',
prefix: '/learn',
},
schema: articleSchema,
Expand Down
2 changes: 2 additions & 0 deletions layers/marketing/content/learn/create-agent-skills.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ updatedAt: 2026-10-05
command: npx skilld run skilld-dev/skilld/generate-project-skill
---

Our [study of 12,141 Skills](/learn/research/skill-md-size-study) measures descriptions, body tokens, and reference files separately.

Write one `SKILL.md` with the procedure you want an Agent to follow.
Keep its instructions independent of Agent-specific features.
Then check discovery and behavior in each Agent you support.
Expand Down
2 changes: 2 additions & 0 deletions layers/marketing/content/learn/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ description: Practical guides for finding, authoring, and shipping agent skills.
---

::card-grid
- [SKILL.md size and descriptions: 12,141 Skills measured](/learn/research/skill-md-size-study)
Original research into context use, descriptions, and references across 2,297 repositories.
- [Create Skills for Claude Code, Codex and Gemini CLI](/learn/create-agent-skills)
Write one `SKILL.md`, check discovery paths, and test the same task across Agents.
- [Browse skills](/skills)
Expand Down
Loading
Loading