Skip to content

feat(demos): record OpenCode writing Skill comparisons - #602

Merged
harlan-zw merged 1 commit into
mainfrom
feat/writing-evals
Oct 9, 2026
Merged

harlan-zw merged 1 commit into
mainfrom
feat/writing-evals

Conversation

@harlan-zw

Copy link
Copy Markdown
Collaborator

🔗 Linked issue

❓ Type of change

  • 📖 Documentation
  • 🐞 Bug fix
  • 👌 Enhancement
  • ✨ New feature
  • 🧹 Chore
  • ⚠️ Breaking change

📚 Description

The writing demos need recorded Skill outputs so readers can compare real rewrites.
Six writing Skills now show three Markdown documents from OpenCode on zai-coding-plan/glm-5.3.
Each receives the same generated seed and shares one rewrite without a Skill.

The runner pins each Skill source, isolates every session, and keeps prompts, transcripts, Markdown, and material checks.
The records replace the shared fixtures on the existing full demo URLs.
The evaluation notes retain meaning changes and explain the limits of this single run per Skill.
Brundlefly uses its published im-not-a-fly source.

Writing evaluation flow from pinned sources through OpenCode to reviewed Markdown and the existing reader

🤖 AI disclosure: Harlan Agent Kit modified this description. My AI open-source policy.

@harlan-github-agent harlan-github-agent Bot added harlan-agent-running An Agent holds a Task on this issue or pull request right now. harlan-agent-review-required Pull request triage requires an adversarial Review for this head commit. labels Oct 9, 2026
@harlan-github-agent

harlan-github-agent Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

🤖 MERGED

Harlan Agent Kit posted this automated review. It is not Harlan's personal review or approval. AI open source policy. Last updated: 2026-10-09 16:51 UTC.

GitHub merged this pull request.

  • Logged (20/100): The pull request's flow diagram is unavailable, so the documented evaluation flow it illustrates is unverified evidence in the PR itself. View code Optional follow-up: Re-attach the diagram or replace it with a link to docs/evals/writing-2026-10-09/README.md, which documents the same flow.
    • Ask an agent to verify and fix this finding
  • Logged (12/100): COPY.md's copy guidance for writing demos now describes the retired fixture regime instead of the recorded demos this change publishes. View code Optional follow-up: Update the 'Writing demo example' row (and GLOSSARY.md's fixture note if needed) to describe the recorded writing demos, or remove the row.
    • Ask an agent to verify and fix this finding

402f3337-69e3-49dd-a96f-da895183774a

Selected findings run after merge and open separate pull requests.

@harlan-zw

Copy link
Copy Markdown
Collaborator Author

Agent verification for this recorded comparison:

OpenCode 1.18.32 ran zai-coding-plan/glm-5.3 for one seed, one no-Skill baseline, and six Skill rewrites.
Each run returned three Markdown documents. All exact code, URL, and identifier checks passed.
The agent inspected all 18 Skill outputs and recorded meaning changes in docs/evals/writing-2026-10-09/README.md.
The outputs remain unedited. This single run does not establish a quality ranking.

Browser checks covered all six full demo URLs, file selection, highlighted README fences, Diff, and Skill footer links.
The API returned one shared original, one shared baseline, and six distinct output sets.
Copy demo link worked. The 375px view had no page overflow. No browser page errors occurred.

Recorded Brundlefly README in the existing demo reader

Recorded PR document at 375px

@harlan-github-agent harlan-github-agent Bot added harlan-agent-ready The automated Review passed every gate on this head commit. and removed harlan-agent-running An Agent holds a Task on this issue or pull request right now. harlan-agent-review-required Pull request triage requires an adversarial Review for this head commit. labels Oct 9, 2026

@harlan-github-agent harlan-github-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Harlan Agent Kit posted this automated review. AI open source policy.

Automated Review: READY.

Read the Review results

@harlan-zw
harlan-zw merged commit 7a339c5 into main Oct 9, 2026
8 checks passed
@harlan-github-agent harlan-github-agent Bot removed the harlan-agent-ready The automated Review passed every gate on this head commit. label Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant