Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .claude/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
review-runs/
__pycache__/
114 changes: 114 additions & 0 deletions .claude/skills/generate-docs/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
---
name: generate-docs
description: Generate or update developer-portal docs from a deployment change (e.g. a helm-charts PR or a new service version) for any service or domain. Discovers what changed and how the services work, verifies the flow live, proposes a change plan, then writes pages and opens a draft docs PR. Use for "document <deployment PR>", "generate docs for the new <service> version", or "/generate-docs".
argument-hint: "<owner/repo#N deployment PR> --env <name> [--namespace <ns>] [--release <r>]"
---

# Generate docs from a deployment change

Docs are the spec readers build against, so what gets documented is the **declared intent**
of the change (chart diff, config, the service's own self-description and code), **verified
live**. Where intent and live behaviour disagree, document the intent and report the mismatch
as a deployment finding. Never bake a deployment bug into the docs.

Nothing here is specific to one domain or protocol. Discover each time; use recipes only as a
head start.

Helper: `python3 .claude/tools/docrev/docrev.py` (`docrev` below; `docrev <cmd> -h`).
Environment setup (config in `~/.claude/review-envs/`, `env check`, `env forward`, token
handling) is the same as in the `review-docs` skill, section 2. Follow it.

## 1. What changed

- `docrev deploy-diff --pr <owner/repo#N>`: new/modified files, added image/tag/route/
dependency keys with line numbers. Read the diff itself for config files it lists
(profiles, mappings, service config): those usually carry the intent.
- `docrev inventory --namespace <ns> --release <r>`: what is actually running (images, ready
replicas), exposed (routes, admission), and configured (configmaps) for the release.
- Build a list of **changed capabilities**, each tied to evidence: a new service or API
version, new/removed fields, new endpoints or operations, new link types, changed auth,
changed limits. Ignore pure infra changes (resources, replicas, probes) unless they alter
behaviour a client sees.

## 2. Discover each service

For every service behind a changed capability:

1. `recipes/` — if a recipe matches (by image name, protocol, or API style), read it first.
2. Self-description, the way a client would find it (try, don't assume):
OpenAPI/Swagger (`/openapi.json`, `/swagger.json`, `/api-docs`, `/docs`), OGC
`GetCapabilities` / `DescribeRecord` / `DescribeFeatureType` / `GetDomain`, `OPTIONS`,
HAL/JSON:API links, GraphQL introspection, index pages. Use `docrev call` for every request.
3. Declared configuration and code: config files from the chart diff; files inside the
running image via `docrev pod-read` (profile/schema definitions, route tables, mapping
files). `pod-read` masks obvious secrets; still never copy credentials, internal hostnames,
or tokens into docs, recipes, or chat.
4. Existing docs for the same service/domain (`docs/**`): the previous version's pages are the
template and the baseline for "what's new".

## 3. Intent vs live

For each capability, compare what the declaration says with what the service does, using
structure rather than values:

- `docrev shape-diff <declared> <live> [--under <element>]`, where either side can be a
saved response, a file, or a doc block (`doc.md:LINE`). E.g. the previous version's doc
example vs a live response shows what's new/removed; the new profile's declared fields vs a
live record shows whether the deployment actually serves them.
- Try the new capability the way a reader would (filter on a new field, follow a new link
type, call a new operation). A declared field that can't be queried, a link type that never
appears, an operation that errors — each is a **deployment finding**.
- Localise every mismatch hop by hop before naming a cause: public route → proxy → backend.
Query the backend directly (`docrev pod-call`, bypasses routes/proxies/auth) and compare
with the same request through the route; check what each hop actually mounts/loads
(`pod-read`, the Deployment's volumes), not only what the ConfigMaps say. Service logs
often state the cause outright (e.g. an undefined DB column). Treat a restart as a test of
a hypothesis, and re-run `env forward` afterwards (forwards die with their pod).
- Before posting a cause, it must be verified; an unverified hypothesis goes out as a
question, not a finding.
- Walk the intended reader flow end to end (search → metadata → data, or whatever the service
implies), chaining values between steps exactly as `review-docs` section 4 describes,
including its safety rules (writes only with per-request approval; downloads via `--range`).

## 4. Change plan (stop for approval)

Present before writing anything:
- Pages to create/update, each with its template page (sibling or previous version) and what
it gets: new sections/steps, table rows with change markers, example requests/responses.
- The reader flow per guide page, as the step list it will have.
- Deployment findings (intent vs live mismatches) with evidence; these are reported, not
documented as behaviour.
- Open questions you can't settle from evidence. Ask; don't pick.

Wait for the user to approve or adjust.

## 5. Write

In a new worktree/branch off the default branch (don't touch the user's working tree):
- Mirror the template page's structure, front matter, tags, admonitions, tabs, and diagram
style. Add new pages to `sidebars.js` next to their siblings.
- Examples: requests use the existing placeholder style (`{SERVICE_URL}`, `<token>`);
responses are captured live, trimmed to what the reader needs, with internal hostnames
replaced by placeholders. Where live output contradicts intent (a deployment finding),
write the example from the intent and leave an HTML comment in the page naming the
finding so reviewers see it.
- Reference tables (fields, enums, parameters) come from the declaration; mark changes vs the
previous version the way existing pages do.
- Keep prose short: step purpose, what to take from the response for the next step, gotchas
found during discovery.

## 6. Verify and open the PR

- Run the `review-docs` procedure on the new/changed pages against the same env. Fix doc
findings; keep deployment/env findings for the report.
- `npm run build` must pass with no broken link/anchor warnings for the new pages (the site
config only warns, so read the output).
- Open a **draft** PR (Conventional Commits title, `docs(<domain>): ...`). Body: capabilities
documented, reader flows, verification result per page, deployment findings with links to
the deployment PR lines. No pasted code; reference paths.

## 7. Learn

If this service kind had no recipe, or the recipe was wrong/incomplete, add or update
`recipes/<kind>.md` (format in `recipes/README.md`). Include only what is generic and
verified; no internal hostnames, namespaces, tokens, or product data.
19 changes: 19 additions & 0 deletions .claude/skills/generate-docs/recipes/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Recipes

Learned notes per service kind, written by `generate-docs` after it documents a service. A
recipe is a head start, never a substitute for discovery: everything in it is re-verified live.

This repo is public. Recipes hold only generic, verified knowledge: no internal hostnames,
namespaces, tokens, credentials, or product data.

Format (`<kind>.md`, kind = protocol or product, e.g. `ogc-csw-pycsw`):

```markdown
# <kind>
Match: <how to recognise it — image name pattern, endpoint, response root>
Self-description: <where the service describes itself>
Declared intent lives in: <config/code files, in the chart or the image>
Reader flow: <typical steps and what each step hands to the next>
Gotchas: <verified pitfalls>
Docs: <existing pages that use it, as templates>
```
24 changes: 24 additions & 0 deletions .claude/skills/generate-docs/recipes/ogc-csw-pycsw.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# ogc-csw-pycsw

Match: image `common/pycsw` or `pycsw`; `POST .../csw` answers `csw:GetRecordsResponse`.

Self-description: `GetCapabilities`, `DescribeRecord`; queryables per profile via `GetDomain`.

Declared intent lives in:
- `mappings.py` in the chart (queryable → DB column),
- `pycsw.cfg` (`profiles=`, `table=`, repository `filter=`),
- the profile code inside the image: `/home/pycsw/pycsw/plugins/profiles/<profile>/` (read with `pod-read`).

Reader flow: `GetRecords` with a filter (tabs for all / id / type / bbox / polygon / point), paging via `startPosition`/`nextRecord`, then take the record's `mc:links` (by `scheme`) into the next service.

Gotchas:
- A pycsw with a new profile/mappings that points at the old records table has been observed serving old-shaped records. Compare a live record's shape with the declared profile (`shape-diff --under <record element>`), and try filtering on a new field: `Invalid PropertyName` means the new profile isn't really served.
- The declared queryables are listed in GetCapabilities under the `<Profile>Queryables` constraint (e.g. `MCDEMQueryables`); comparing that list and the `GetRecords` DCP href with the expected version quickly shows whether the route reaches the intended pycsw instance.
- Per-instance nginx in front of pycsw (unified chart) mounts its `location.conf` (`uwsgi_pass <backend>`) from a ConfigMap named in `nginx.extraVolumes`; when a second instance doesn't override it, it proxies to the first instance's backend.
- Mappings pointing at columns the table lacks surface as `Invalid query syntax` in CSW and `UndefinedColumn` in the pycsw log; filters on shared columns keep working, which hides the problem.
- A declared queryable isn't necessarily usable everywhere: e.g. `mc:boundingBox` was rejected in spatial filters while `mc:BoundingBox` / `ows:BoundingBox` worked. Test each documented name in the filter types the docs use.
- The profile's bundled XSD (DescribeRecord) can be stale (seen describing another record type); don't take field types from it without checking.
- Polygon filters must be GML3 (`gml:exterior` / `gml:LinearRing` / `gml:posList`); GML2 `outerBoundaryIs`/`coordinates` is rejected with `Missing gml:posList`.
- Errors come back as `ows:ExceptionReport` with HTTP 200.

Docs: `docs/MapColonies/*/Services/catalog/profile_v*.md`, Step 1 of the DEM/3D guides.
16 changes: 16 additions & 0 deletions .claude/skills/generate-docs/recipes/ogc-wcs-geoserver.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# ogc-wcs-geoserver

Match: image `*geoserver*`; `GET .../wcs?request=GetCapabilities` answers `wcs:Capabilities`.

Self-description: `GetCapabilities` (formats, CRSs, interpolations, `wcs:CoverageId`s), `DescribeCoverage` (envelope, axis labels, grid, nil values).

Declared intent lives in: chart env (`PROXY_BASE_URL`, extensions), the GeoServer data dir on the PVC (workspace, output limits), the ingestion/publishing service that names coverages.

Reader flow: GetCapabilities → pick coverage → DescribeCoverage (take `srsName`, `axisLabels`) → GetCoverage (whole / `subset=` per axis / format / optional scaling, `outputCRS`, interpolation).

Gotchas:
- Coverage ids are prefixed with the workspace (`<ws>__<name>`); the prefix is optional in requests. Verify the naming rule the docs claim against real ids.
- Every `xlink:href` in capabilities is built from `PROXY_BASE_URL`; check those hosts resolve to a working route, since clients like QGIS, GDAL and OWSLib follow them.
- Output size limit errors are `ows:ExceptionReport` with HTTP 500; the limit is per-environment config.

Docs: `docs/ogc/protocols/ogc-wcs.md`, Steps 2–3 of the DEM height extraction guide.
16 changes: 16 additions & 0 deletions .claude/skills/generate-docs/recipes/s3-download-gateway.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# s3-download-gateway

Match: image `common/nginx-s3-gateway`; serves objects of an S3 bucket under a route path.

Self-description: none; object paths come from catalog links (e.g. `scheme="Download"`).

Declared intent lives in: chart values (`route.path`, bucket, `authorization.opa`, directory listing flag).

Reader flow: catalog record → `Download` link → `GET <link>?token=...`.

Gotchas:
- Verify with `call --range 1024` rather than downloading whole files.
- Directory paths return 404 when listing is disabled, which is expected.
- Check the no-token response: it should be 401/403, and a 500 is a deployment finding.

Docs: `docs/MapColonies/DEM/Services/download/README.md`.
Loading