Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/release-allowed-signers
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
release@openclaw.ai ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAII9XsaCcr8TInPnHcuTVfvXXcsoUFrOE7menfbEIHFW9 steipete@gmail.com
2 changes: 1 addition & 1 deletion .github/scripts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ Both complete CLI suites remain required, against the published baseline and
the exact candidate head. Selected-app manual qualification retains baseline
`26a574e53b485cd2191df603f02d17fab6d3c4af` and its existing app pins.

The Linux job has a 20-minute ceiling. It prepares dependencies with Go 1.27.2
The Linux job has a 20-minute ceiling. It prepares dependencies with Go 1.27.3
and the public Go proxy, then runs baseline and candidate serially with
`GOPROXY=off`. Each uses native `lsof`, private HOME/XDG/temp directories, a
private test modfile, and a network/PID namespace with loopback only. The
Expand Down
2 changes: 1 addition & 1 deletion .github/scripts/downstream-compat.sh
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ fi
for tool in go git lsof tar sha256sum sudo unshare ip setpriv timeout python3; do
command -v "$tool" >/dev/null || { echo "missing required tool: $tool" >&2; exit 2; }
done
[[ $(go env GOVERSION) == go1.27.2 ]] || { echo "Go 1.27.2 is required" >&2; exit 2; }
[[ $(go env GOVERSION) == go1.27.3 ]] || { echo "Go 1.27.3 is required" >&2; exit 2; }
sudo -n true
[[ $(setpriv --help) == *"--pdeathsig"* ]] ||
{ echo "setpriv --pdeathsig keep is required for descendant cleanup" >&2; exit 2; }
Expand Down
15 changes: 11 additions & 4 deletions .github/scripts/release-preflight.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,19 @@ module.exports = async function preflight({ github, context, version, dryRun = f
throw new Error(`${tag} already has a ${kind}; refusing to rebuild an existing release version. Use a new patch version, or reconcile the original run without rebuilding its verified payload.`);
};

// Tags are public Go module versions even without a published GitHub release.
// The shared workflow creates the tag before creating any draft or assets.
await requireAbsent(() => github.rest.git.getRef({
// The shared workflow verifies the fleet signature before building or publishing.
const ref = await github.rest.git.getRef({
...context.repo,
ref: `tags/${tag}`,
}), 'Git tag');
});
if (ref.data.object.type !== 'tag') throw new Error(`${tag} must be an annotated fleet-signed tag`);
const annotated = await github.rest.git.getTag({
...context.repo,
tag_sha: ref.data.object.sha,
});
if (annotated.data.object.type !== 'commit' || annotated.data.object.sha !== context.sha) {
throw new Error(`${tag} must point directly to the release workflow commit`);
}
await requireAbsent(() => github.rest.repos.getReleaseByTag({
...context.repo,
tag,
Expand Down
38 changes: 25 additions & 13 deletions .github/scripts/release-preflight.test.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -3,43 +3,47 @@ const { readFileSync } = require('node:fs');
const { test } = require('node:test');
const preflight = require('./release-preflight.cjs');

const context = { repo: { owner: 'openclaw', repo: 'crawlkit' } };
const context = { repo: { owner: 'openclaw', repo: 'crawlkit' }, sha: 'a'.repeat(40) };
const missing = () => { throw Object.assign(new Error('Not Found'), { status: 404 }); };

function fixture({ tag = missing, release = missing } = {}) {
function fixture({ tag = () => ({ data: { object: { type: 'tag', sha: 'b'.repeat(40) } } }), annotated = () => ({ data: { object: { type: 'commit', sha: context.sha } } }), release = missing } = {}) {
const calls = [];
return {
calls,
// Only read endpoints exist: mutations are never part of preflight.
github: { rest: {
git: { getRef: async (args) => { calls.push(['tag', args]); return tag(); } },
git: {
getRef: async (args) => { calls.push(['tag', args]); return tag(); },
getTag: async (args) => { calls.push(['annotated', args]); return annotated(); },
},
repos: { getReleaseByTag: async (args) => { calls.push(['release', args]); return release(); } },
} },
};
}

for (const version of ['0.14.8', 'v0.14.8', '0.14.8-rc.1', 'v0.14.8+build.1']) {
test(`unused ${version} reaches the shared release workflow`, async () => {
test(`annotated ${version} at the workflow commit reaches signature verification`, async () => {
const { github, calls } = fixture();
await preflight({ github, context, version });
const tag = `v${version.replace(/^v/, '')}`;
assert.deepEqual(calls, [
['tag', { ...context.repo, ref: `tags/${tag}` }],
['annotated', { ...context.repo, tag_sha: 'b'.repeat(40) }],
['release', { ...context.repo, tag }],
]);
});
}

for (const kind of ['tag', 'commit']) {
test(`existing ${kind} stops before rebuilding, even without a GitHub release`, async () => {
for (const kind of ['commit', 'tree', 'blob']) {
test(`non-annotated ${kind} stops before building`, async () => {
const { github, calls } = fixture({ tag: () => ({ data: { object: { type: kind } } }) });
await assert.rejects(preflight({ github, context, version: 'v0.14.7' }), /v0\.14\.7 already has a Git tag/);
await assert.rejects(preflight({ github, context, version: 'v0.14.7' }), /must be an annotated fleet-signed tag/);
assert.equal(calls.length, 1);
});
}

for (const draft of [false, true]) {
test(`existing ${draft ? 'draft' : 'public release'} blocks even if its tag is missing`, async () => {
test(`existing ${draft ? 'draft' : 'public release'} blocks publication`, async () => {
const { github } = fixture({ release: () => ({ data: { id: 123, draft } }) });
await assert.rejects(preflight({ github, context, version: '0.14.7' }), /already has a GitHub release/);
});
Expand All @@ -51,10 +55,10 @@ test('the halted v0.14.7 state cannot reach build or draft creation', async () =
release: () => ({ data: { id: 370802867, draft: false } }),
});
await assert.rejects(preflight({ github, context, version: '0.14.7' }), /refusing to rebuild/);
assert.equal(calls.length, 1);
assert.equal(calls.length, 3);
});

for (const endpoint of ['tag', 'release']) {
for (const endpoint of ['tag', 'annotated', 'release']) {
for (const status of [401, 403, 429, 500]) {
test(`${endpoint} HTTP ${status} fails closed`, async () => {
const error = Object.assign(new Error('API unavailable'), { status });
Expand Down Expand Up @@ -87,11 +91,18 @@ test('dry runs still reject invalid versions', async () => {
assert.deepEqual(calls, []);
});

test('an explicit publication still rejects an existing version', async () => {
const { github } = fixture({ tag: () => ({ data: { object: { type: 'tag' } } }) });
await assert.rejects(preflight({ github, context, version: 'v0.17.0', dryRun: false }), /already has a Git tag/);
test('publication requires a pre-created tag', async () => {
const { github } = fixture({ tag: missing });
await assert.rejects(preflight({ github, context, version: 'v0.17.0', dryRun: false }), /Not Found/);
});

for (const object of [{ type: 'commit', sha: 'c'.repeat(40) }, { type: 'tag', sha: context.sha }]) {
test(`rejects a tag that does not directly freeze the workflow commit (${object.type})`, async () => {
const { github } = fixture({ annotated: () => ({ data: { object } }) });
await assert.rejects(preflight({ github, context, version: '0.17.2' }), /must point directly to the release workflow commit/);
});
}

for (const dryRun of ['true', 'false', 1, null]) {
test(`non-boolean dryRun ${JSON.stringify(dryRun)} fails closed`, async () => {
const { github, calls } = fixture();
Expand All @@ -109,4 +120,5 @@ test('workflow gates the reusable job and serializes preflight through publicati
assert.match(workflow, /DRY_RUN: \$\{\{ inputs.dry_run \}\}/);
assert.match(workflow, /await preflight\(\{ github, context, version: process.env.RELEASE_VERSION, dryRun: process.env.DRY_RUN === 'true' \}\)/);
assert.match(workflow, /dry-run: \$\{\{ inputs.dry_run \}\}/);
assert.match(workflow, /require-signed-tag: true/);
});
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ jobs:
persist-credentials: false
- uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
with:
go-version: '1.27.2'
go-version: '1.27.3'
cache: false
- name: Compare pinned downstream CLI suites
run: bash candidate/.github/scripts/downstream-compat.sh candidate baseline gitcrawl
Expand Down Expand Up @@ -110,7 +110,7 @@ jobs:
persist-credentials: false
- uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
with:
go-version: '1.27.2'
go-version: '1.27.3'
cache: false
- name: Compare pinned app tests, builds, and read-only smokes
env:
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/release-unified.yml
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,7 @@ jobs:
version: ${{ inputs.version }}
dry-run: ${{ inputs.dry_run }}
repository-type: openclaw
require-signed-tag: true
checksum-filename: checksums.txt
nfpm: auto
stable-identifier: org.openclaw.crawlctl
Expand Down
7 changes: 4 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,10 +76,11 @@ then consume it from the app.
## Release Model

Release only through `.github/workflows/release-unified.yml`. The shared
`openclaw/release-workflows` pipeline owns tag creation, Developer ID signing,
`openclaw/release-workflows` pipeline verifies fleet-signed tags and owns Developer ID signing,
notarization, independent artifact verification, and GitHub Release
publication. Never create or sign release tags locally, publish locally built
artifacts, or handle release credentials outside the workflow. There is no npm,
publication. After a successful dry run, create and verify an annotated release
tag with the fleet signing key before dispatching publication. Never publish locally built
artifacts or handle binary-signing credentials outside the workflow. There is no npm,
PyPI, or Homebrew publish step for `crawlkit`.

Use patch tags for narrow fixes and minor tags for broader shared crawler or TUI
Expand Down
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,14 @@

## Unreleased

## 0.17.2 - 2026-10-20

**Highlights:** Rebuild with Go 1.27.3 to fix the reachable crypto/x509 certificate-validation vulnerability.

### Security

- Build with Go 1.27.3 to fix the reachable crypto/x509 certificate-validation vulnerability GO-2026-6809 while retaining the Go 1.27.0 minimum; require fleet-signed release tags.

## 0.17.1 - 2026-10-20

**Highlights:** Refreshed security analysis and dependency checks, with more reliable Crabbox validation and macOS release preflight.
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Provider APIs, authentication, schemas, privacy filters, and user-facing command

`crawlkit` requires Go 1.27.0 or newer.

Development builds prefer Go 1.27.2 through the `toolchain` directive in `go.mod`.
Development builds prefer Go 1.27.3 through the `toolchain` directive in `go.mod`.

Add the package you need to a Go module. For the quick start below:

Expand Down
28 changes: 17 additions & 11 deletions docs/publishing.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,10 @@

`crawlkit` is a Go library with the optional `crawlctl` CLI. Releases use the
fleet `release-go-cli` workflow from `openclaw/release-workflows`. The workflow
owns tag creation, Developer ID signing, notarization, independent artifact
verification, and GitHub Release publication. Do not create release tags or
handle signing credentials locally.
verifies fleet-signed tags and owns Developer ID signing, notarization, independent
artifact verification, and GitHub Release publication. Create and verify the
annotated tag with the fleet release key only after a successful dry run. Binary
signing credentials remain in the workflow.

Published versions and signed CLI assets are listed on
[GitHub Releases](https://github.com/openclaw/crawlkit/releases/latest) and the
Expand Down Expand Up @@ -34,7 +35,7 @@ signing identity is `Developer ID Application: OpenClaw Foundation
(FWJYW4S8P8)`. The workflow signs each native Mach-O with the hardened runtime,
submits it for notarization, and independently verifies both architectures
before publication. Both native macOS binaries require macOS 13.0 or later,
matching the Go 1.27 deployment target. Before the shared workflow creates a tag,
matching the Go 1.27 deployment target. Before the shared workflow builds artifacts,
the caller builds both architectures with CGO disabled, checks each
`LC_BUILD_VERSION` using `otool -l`, and confirms notarization access with a
read-only `xcrun notarytool history`. crawlkit has no universal macOS archive, nFPM package, or
Expand Down Expand Up @@ -67,17 +68,22 @@ The default remains a real release, which requires an unused version.
```

3. Merge the release-preparation PR and confirm its exact merge commit is green.
4. Dispatch the unified workflow from the current protected `main` head:
Run the dry run above with the new version, then create the annotated version
tag on that commit with the fleet release key. Verify it with
`git -c gpg.ssh.allowedSignersFile=.github/release-allowed-signers verify-tag vX.Y.Z`
before pushing it.
4. Dispatch the unified workflow from the same protected `main` head:

```bash
gh workflow run release-unified.yml --repo openclaw/crawlkit -f version=X.Y.Z
```

5. Watch the exact workflow run through publication. A read-only preflight
rejects existing tags and releases before the shared workflow runs. A
requires an annotated tag pointing directly to the workflow commit and rejects
existing releases before the shared workflow verifies the tag signature. A
repository-wide workflow lock spans preflight through publication, so queued
dispatches (including `X.Y.Z` and `vX.Y.Z`) recheck state after the previous
run finishes. The shared workflow creates an annotated version tag, builds all four native archives, signs and
run finishes. The shared workflow freezes the verified tag, builds all four native archives, signs and

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Document retries for tag-only release failures

When a live run fails before creating a draft or GitHub Release and main remains at the tagged SHA, the changed preflight now accepts the existing annotated tag and the reusable @v1 workflow reuses it as the frozen retry target. However, the recovery guidance at lines 107–110 still states that every fresh dispatch after a tag exists requires a new version. Following that guidance unnecessarily abandons a valid tagged module version; distinguish tag-only failures, which can be redispatched from the same head, from failures that already created a draft or release.

Useful? React with 👍 / 👎.

notarizes the two macOS binaries, verifies the immutable draft independently
on Apple Silicon and Intel, and publishes only the verified bytes.
6. Confirm the GitHub Release notes match the dated changelog section. Download
Expand All @@ -100,10 +106,10 @@ The default remains a real release, which requires an unused version.

Never delete or force-update a release tag or replace published assets. A Git
tag is already a public Go module version, even when GitHub publication fails.
Fresh dispatches require a new version once a tag exists; this also blocks
leftover drafts because the shared pipeline freezes the tag before drafting.
The release lookup additionally rejects a visible release whose tag is missing.
API failures other than an explicit 404 stop preflight.
Publication requires a fleet-signed tag on the exact workflow commit and no
existing draft or public release. The shared pipeline freezes that tag before
drafting. Tag lookup failures stop preflight; only an explicit 404 for the
release lookup is treated as an unused release version.

After a failure, inspect the exact run, tag, release IDs, and retained artifacts
before taking action. Do not rerun all jobs or redispatch a published version:
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ module github.com/openclaw/crawlkit

go 1.27.0

toolchain go1.27.2
toolchain go1.27.3

require (
filippo.io/age v1.3.2
Expand Down
Loading