feat(report): publish a Taskless report to tskl.es

[theCodeDrift]

Stack (root → tip):

• feat(detect): report ast-grep, Vale, host tools, and agent hints #495
  • refactor(rules): rename rules/report to rules/check-summary #496
    • ➡️ feat(report): publish a Taskless report to tskl.es #497 (you are here)
      • feat(onboard): one question, then rules, a report, and a CI offer #498
        • docs(openspec): archive streamlined-onboarding-report #499

Third in the stack. Adds taskless report:

• No flags: an offline JSON draft. It has the scope from origin, every rule under .taskless/rules/ with its test status and first fixtures, the default projection inputs, and missing[], the facts the agent must supply.
• --from <facts.json>: the agent supplies session facts, never a ReportV1. The CLI then merges them (its own values win), refuses with REPORT_INSUFFICIENT below three working Taskless rules (sg/vale, or legacy rules the agent wrote), computes perMonth, and withholds any fixture a pattern matches. It refuses REPORT_REDACTION for an agent-written title that matches, validates against the vendored schema, publishes, writes .taskless/reports/<stamp>.json, and prints a templated summary. The delete URL never reaches stdout or telemetry.
• --delete <record>: sends an HTTP DELETE to the recorded deleteUrl, and removes the record only on success.

ReportV1 is vendored from https://tskl.es/api/v1/schema (generate:report-schema) and validated with z.fromJSONSchema(). That covers the schema's own example and 11 counter-examples, including unknown keys, scope variants, and fixture length. The publish URL, size cap, and fixture limit are all read from the document. Retry follows x-taskless: retry only a 502, never after a timeout (REPORT_PUBLISH_UNKNOWN), and surface a 429.

Delete contract: vendored from the schema's x-taskless.delete. It's a DELETE to the deleteUrl. A 204 removes the record, and the confirmation notes caches may serve the page for up to a day. A 404 not_available isn't retried, exits with REPORT_NOT_AVAILABLE, and keeps the record, since a just-published report can 404 before it propagates. A test pins the vendored contract, so a refresh that changes it fails loudly.

Also adds the report agent topic, migration 11 (.taskless/reports/, gitignored), and the bump of this repo's own .taskless/ to v11.

delete is a flag rather than a subcommand: citty dispatches the first argument without a leading - as a subcommand, so report --from <path> would fail as "Unknown command".

[theCodeDrift]

@claude /review

[github-actions]

Claude finished @theCodeDrift's task in 3m 59s —— View job

---

Review: taskless report (PR #497)

Review mode: incremental — read 0 prior review thread(s) before reviewing. .prior-review.json contained no resolved/unresolved review threads and no prior review bodies — only the @claude /review trigger comment itself — so this is the first full review of this PR; nothing is being treated as already addressed.

Scope covered: full diff via gh pr diff (schema vendoring, draft/assemble/redact/publish/records/summary logic, command wiring, migration 0011, agent topic, and all four new/updated test files), plus whole-file reads of report/assemble.ts, report/redact.ts, schemas/report-facts.ts, report/publish.ts, report/records.ts, report/draft.ts, report/types.ts, report/schema.ts, report/summary.ts.

---

[New] Redaction screening skips legacy[].tool, legacy[].language, and cloud[].language — free text that is published verbatim

screenStrings (packages/cli/src/report/assemble.ts:178-195) only scans the label and each rule's title:

for (const group of ["rules", "legacy", "cloud"] as const) {
  for (const rule of facts[group]) {
    fields.push([`${group}[${rule.id}].title`, rule.title]);
  }
}

But mergeRules (same file, ~line 85 and ~105) writes legacy.tool, legacy.language, and cloud.language straight onto the published ReportRule with no scan at all:

tool: legacy.tool,
language: legacy.language,
...
language: cloud.language,

These are agent-supplied free text (schemas/report-facts.ts:70 tool: z.string().regex(/^[\w@./+-]+$/).max(64), lines 75 & 86 language: z.string().min(1).max(40)) — unconstrained enough to hold an email address (@ and . are both allowed, no space requirement) or a short token/hostname-looking string, and unlike fixture code (redactFixtures) they go through no scanText pass and no REPORT_REDACTION check. Since tskl.es "performs no redaction of its own" (per the PR body and report/redact.ts's own docstring: "what this misses is published"), anything an agent puts in these two fields reaches the public page unmasked.

This is also untested: report.test.ts's only REPORT_REDACTION case is an email in a title (line ~3367); the legacy/cloud test fixtures (lines ~3261-3278, ~3302-3328) use innocuous tool: "eslint" / language: "TypeScript" values and never exercise a hostile value in either field.

Fix: extend screenStrings's fields list to also push legacy[id].tool and {group}[id].language for legacy/cloud entries (and add a test asserting an email in cloud[].language is refused).

---

[New] Minor: report --delete <record> can resolve outside .taskless/reports/

recordPath (packages/cli/src/report/records.ts, the resolve(cwd, record) branch) accepts any relative path, not just a bare file name, so --delete ../../somewhere/file.json reads and — on a successful remote delete — rms a file outside .taskless/reports/. Given this is a local CLI run by the user/agent on their own machine (same trust boundary as every other -d/file-path flag in this codebase), this isn't a cross-trust-boundary vulnerability, just a bit more permissive than the README.md stub ("Delete a report with taskless report --delete <file>") implies. Low severity — flagging for awareness rather than requesting a change.

---

Everything else checked, no issues found

• Gate logic (enforceGate, MINIMUM_TASKLESS_RULES): correctly counts only cli/legacy tiers, excludes fail/refused statuses, and cloud candidates never count — matches the spec and is well covered by tests (insufficient-rules, failing-rule-excluded, cloud-doesn't-rescue cases).
• Fixture redaction (redactFixtures/scanText): withhold-whole-not-mask is implemented correctly; counts from scanText and the agent's own facts.redaction.counts are summed without double-counting; HOST/IP patterns correctly exclude example.com/localhost/loopback addresses.
• Publish retry contract (publish.ts): 502 retried (capped at 3 attempts), 429 surfaced as REPORT_RATE_LIMITED without retry, network/timeout errors surfaced as REPORT_PUBLISH_UNKNOWN without retry, 400/413 → REPORT_INVALID with service details — matches the vendored x-taskless.publish contract and is exercised against a real local HTTP server in report.test.ts.
• Delete contract: 404 → REPORT_NOT_AVAILABLE, not retried, record kept; 204 removes the record; the vendored-contract pin test ("is a DELETE to the deleteUrl...") will catch a schema drift as intended.
• Secrecy of the delete URL: never printed in either --json or plain output (summary.ts, report.ts), confirmed by both the end-to-end test and the dedicated telemetry test asserting no event payload contains the URL, delete path, title, or fixture text.
• Migration 0011: idempotent, anchors /reports/ with a leading slash consistent with migration 0007's convention, leaves existing records/edited README alone on rerun, and is verified to actually be git-ignored via git check-ignore.
• --from/--delete mutual exclusivity: checked and refused as INVALID_INPUT before ensureTasklessDirectory, with a dedicated test.
• mergeRules: symmetric validation (facts naming an unknown rule vs. a draft rule missing facts) both correctly refused as REPORT_INVALID, duplicate-id check across rules/legacy/cloud in place.

One thing I could not verify from the diff/current files alone: whether scripts/fetch-report-schema.ts's vendored copy matches the current live https://tskl.es/api/v1/schema (no network access in this review) — the title/property sanity check in the fetch script and the contract-pin test are the right safeguards for that, and both are present.

[theCodeDrift]

Re: @github-actions — "Review: taskless report (PR #497)"
#497 (comment)

1. Redaction screening skips legacy[].tool, legacy[].language, cloud[].language: fixed in 5c40a3a. screenStrings now scans those three, and also every rule's id, because the id schema allows dots, so a legacy or cloud id can be a host name (for example api.acme.io) and is published verbatim. Tests landed in 06150e8: one case per field, each asserting REPORT_REDACTION names the field. The spec gains a scenario for it. The requirement already said every agent-supplied string is screened, so this brings the code in line with it.
2. --delete <record> can resolve outside .taskless/reports/: fixed in 06150e8. The finding was raised for awareness only, but the spec already says the record is read from under .taskless/reports/, and a successful delete rms the file. A record whose resolved path is not directly in that directory, or is not a .json file, is now refused with INVALID_INPUT before anything is read or sent. A bare file name or .taskless/reports/<name>.json still works. Covered by a CLI test, and the spec gains a scenario for it.

— AI Coding Agent

[theCodeDrift]

@claude /review

[github-actions]

Claude finished @theCodeDrift's task in 3m 39s —— View job

---

Review: taskless report (PR #497)

Review mode: incremental — read 2 prior review finding(s) before reviewing (surfaced as top-level comments, not inline review threads — .prior-review.json's reviewThreads was empty). Both carry the *— AI Coding Agent* addressed-marker in theCodeDrift's reply and claim specific fix commits (5c40a3a, 06150e8). I verified both against the current code rather than taking the marker on faith:

• Redaction screening skipped legacy[].tool, legacy[].language, cloud[].language — confirmed fixed. screenStrings (packages/cli/src/report/assemble.ts:178-209) now scans those three fields, plus every rule's id across rules/legacy/cloud (ids can contain dots and read as host names). Covered by a parameterized test (report.test.ts:297-347) asserting REPORT_REDACTION names the right field for each of four cases.
• --delete <record> could resolve outside .taskless/reports/ — confirmed fixed. recordPath (packages/cli/src/report/records.ts:59-72) now requires the resolved path's parent directory to be exactly .taskless/reports/ and the file to end in .json, refusing with INVALID_INPUT before any read/write. Covered by report.test.ts:699-709 (../outside.json, /tmp/outside.json, notes.txt, all refused).

Not repeating either — both are closed.

Scope of this pass: full diff via gh pr diff, plus whole-file reads of every file under packages/cli/src/report/, schemas/report-facts.ts, commands/report.ts, filesystem/migrations/0011-reports-directory.ts, types/errors.ts, util/json-input.ts, the vendored report-v1.schema.json (cross-checked against types.ts and assemble.ts for required/optional field agreement), and the four report-related test files.

I found nothing NEW since the last review. Specifically checked and found correct:

• enforceGate/MINIMUM_TASKLESS_RULES: only cli/legacy count, cloud never does, disqualifying statuses (fail/refused) excluded, legacy rules are hardcoded not-run so they always count (matches the openspec wording change in this PR: "in either tier only those whose tests are neither fail nor refused").
• mergeRules: local draft rules always publish the CLI's own id/engine/language — the facts file's rules[].id/title is matched by id but never substitutes a CLI-known field, so the (slightly redundant) screening of rules[].id in screenStrings can never actually fire against anything other than an id the CLI itself already created as a directory name — harmless, not a gap.
• redactFixtures: withhold-whole-not-mask, counts from scanText and the agent's own facts.redaction.counts summed from disjoint sources (no double-counting since they cover different fixtures).
• publish.ts: 502 retried up to MAX_ATTEMPTS (3), 429 → REPORT_RATE_LIMITED no retry, network/timeout → REPORT_PUBLISH_UNKNOWN no retry, 400/413 → REPORT_INVALID; matches the vendored x-taskless.publish contract and is exercised against a real local HTTP server.
• deletePublishedReport: 404 → REPORT_NOT_AVAILABLE, not retried, record kept on failure; the vendored-contract pin test (report.test.ts:110-129) will catch a schema drift.
• records.ts: delete URL never printed in either --json or plain output; confirmed by the dedicated telemetry test asserting no captured event payload contains the URL, delete path, title, or fixture text.
• Migration 0011: idempotent, anchors /reports/ with a leading / per the 0007 convention, leaves existing records/edited README alone on rerun, verified git-ignored via git check-ignore.
• --from/--delete mutual exclusivity checked and refused as INVALID_INPUT before ensureTasklessDirectory, with a dedicated test.
• Vendored schema (report-v1.schema.json) required/optional fields (history optional at top level, commentsPrevented optional under perMonth, legacy optional in PerTier) all agree with the hand-written ReportV1 type in types.ts and with what assembleReport actually produces.

One thing I still cannot verify from the diff/current files alone (no network access in this review, same as last time): whether scripts/fetch-report-schema.ts's vendored copy matches the current live https://tskl.es/api/v1/schema. The title/property sanity check in the fetch script and the contract-pin test remain the right safeguards for that.