docs: refresh docs

[sentry-junior]

Scope: whole repo, no audience filter. Part of a devinfra docs refresh.

Markdown line count: 295 before, 324 after. README.md went from 295 to 262 lines. The rest is the new AGENTS.md (21), docs/testing.md (31), and docs/releasing.md (10). The net growth comes from documenting render() config keys, rollback keys, and the build-time asserts. None of these were documented before, and pipeline authors hit all of them.

Files

• README.md: now covers users only. It has install, output-files, a config reference for pipedream.render(), generated pipeline names, the build-time rules for pipeline_fn, env var cascading, PIPEDREAM_GROUP_REGIONS, rollback, and the tasks and stages helpers. Development, Testing, and Release Process moved out.
• AGENTS.md (new): a short router with commands, invariants, and a map to the docs.
• docs/testing.md (new): fixtures and goldens, *.failing.jsonnet, and how to test against GoCD or a service. This content came from the README.
• docs/releasing.md (new): release and rollout steps, from the README.
• libs/pipedream.libsonnet: deleted one narration comment (// Apply transform to all stages). Nothing else in source changed.

Removed: the link to a dev pipedream example repo, which returns 404, and its pinned commit link. Also removed the make vs make test duplication, since test is the default goal and both run fmt, lint, and test.

Corrected claims

• The README's main pipedream example failed to render, with Pipeline-level attributes differ across regions in group. Region 'customer-2' differs. I confirmed this by running jsonnet on the old snippet. The cause is region: region at the pipeline level, which the assert at libs/pipedream.libsonnet:307 rejects for the multi-region st group. The new example renders, and I checked its output.
• auto_pipeline_progression said "Set to true if you want each pipeline to require manual approval". The code does the opposite: false adds the ready and manual wait stages (pipedream.libsonnet:466).
• The install pin was v2.15.0. The latest release is v3.0.10 (gh release list), and the README now points readers to the releases page.
• The PIPEDREAM_GROUP_REGIONS example listed customer-1,customer-2,customer-4,customer-7. The st group is customer-1,customer-2,customer-7 (libs/getsentry.libsonnet:16).
• The PIPEDREAM_GROUP_REGIONS override has undocumented preconditions, and the README now lists them. It only works in multi-region groups, only wraps script tasks, and only acts when the job sets SENTRY_REGION. The library never sets SENTRY_REGION (deploy_target_gate, wrap_task, pipedream.libsonnet:35-51,431-432).
• The README said materials is what "the pipelines watch for changes". The config materials only goes to the deploy-<name> trigger pipeline (pipedream.libsonnet:107-114). Group pipelines use the materials from pipeline_fn.
• The gocd-stages example used gocdtasks without importing it. The README also left out the fetch_materials option (gocd-stages.libsonnet:21).

Could not verify

• That yq is needed to use these libraries (brew install ... yq). Nothing in this repo uses yq, so I left the claim unchanged.
• make gocd in consuming repos. I only checked snuba's Makefile.
• The steps for testing against a dev GoCD config repo. I kept them as a generic how-to.

Code looks wrong rather than the doc (comments not changed)

• pipedream.libsonnet:492-494: the comment says the gate validates a "user-supplied DEPLOY_TARGET_REGION". The variable is PIPEDREAM_GROUP_REGIONS, and the gate compares it against SENTRY_REGION.
• pipedream.libsonnet:579-582: the comment says --ext-var=output-files=true, but the flag is --ext-code. It also says the single-file shape is { pipelines: [ ... ] }, but the code returns { format_version, pipelines: { <name>: ... } }.
• pipedream.libsonnet:15: the header describes render(config, pipeline_fn) and leaves out the parallel param.
• gocd-pipelines.libsonnet:87-88: the comment says pipelines_to_object returns { <pipeline name>: <definition> }. The function also wraps the result in format_version and pipelines.

Candidates to enforce

• README code examples aren't rendered in CI. This is how the broken example went unnoticed. Rendering the example as a fixture would catch it.
• npm run lint (eslint and prettier, which also checks Markdown) runs in CI but not in make.

Skipped

• libs/bash/*.sh comments: the scripts are embedded through importstr into the rendered rollback pipelines. Editing a comment would change the goldens.
• Test fixtures and goldens.

Checks run

• npx prettier --check on the changed Markdown, and npm run lint: pass.
• make (jsonnetfmt, jsonnet-lint, ava) with go-jsonnet v0.20.0: 57 tests pass, and the working tree was clean afterward.
• Rendered the README jsonnet examples against libs/.
• Ran test -e on every relative link and backticked path in the changed docs. I also checked that the external links return 200.

via @joshuarli.

--

View Junior Session [Sentry]