Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
97 changes: 97 additions & 0 deletions .github/workflows/index.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
# Builds index.json — the list of macros with versions, file sizes and
# SHA-256 hashes that OpenSAK's Macro manager reads — after every push to
# main, and publishes it on the "index" branch:
#
# https://raw.githubusercontent.com/OpenSAK-Org/OpenSAK-Macros/index/index.json
#
# The index branch has no history of main: it holds only index.json (and a
# README saying where it comes from), one commit per change. Nobody edits it
# by hand; a run that gives the same file pushes nothing.
#
# Runs scripts/build_index.py from main (only pushes to main trigger it), so
# what lands in the index went through the pull request checks first. The
# script reads git objects only; no macro code is run.

name: Index

on:
push:
branches: [main]
workflow_dispatch:

permissions: {}

concurrency:
group: index
cancel-in-progress: false

env:
INDEX_BRANCH: index
PARSER_URL: https://raw.githubusercontent.com/OpenSAK-Org/OpenSAK/%s/src/opensak/macro/metadata.py

jobs:
index:
name: Build and publish index.json
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v7
with:
ref: ${{ github.event.repository.default_branch }}
fetch-depth: 0 # the versions come from main's history
persist-credentials: false

- uses: actions/setup-python@v7
with:
python-version: "3.12"

- name: Get OpenSAK header parser
run: |
mkdir -p "$RUNNER_TEMP/tools"
ref=$(grep -v '^#' .github/opensak-ref | head -n 1 | tr -d '[:space:]')
curl -fsSL -o "$RUNNER_TEMP/tools/macro_metadata.py" "$(printf "$PARSER_URL" "$ref")"

- name: Build index.json
run: |
mkdir -p "$RUNNER_TEMP/out"
python -I scripts/build_index.py --repo . --ref HEAD \
--metadata "$RUNNER_TEMP/tools/macro_metadata.py" \
--repository "$GITHUB_REPOSITORY" \
--output "$RUNNER_TEMP/out/index.json"

- name: Publish on the index branch
env:
GH_TOKEN: ${{ github.token }}
run: |
out="$RUNNER_TEMP/out"
cat > "$out/README.md" <<'EOF'
# Macro index

`index.json` lists the macros on `main` with their versions, files,
sizes and SHA-256 hashes, for OpenSAK's Macro manager. It is built by
`.github/workflows/index.yml` after every push to `main`; don't edit
this branch by hand. The format is described in MAINTAINERS.md on
`main`.
EOF

remote="https://x-access-token:${GH_TOKEN}@github.com/${GITHUB_REPOSITORY}.git"
work="$RUNNER_TEMP/index-branch"
if git ls-remote --exit-code --heads "$remote" "$INDEX_BRANCH" >/dev/null; then
git clone -q --depth 1 --branch "$INDEX_BRANCH" "$remote" "$work"
else
git init -q -b "$INDEX_BRANCH" "$work"
git -C "$work" remote add origin "$remote"
fi
cp "$out/index.json" "$out/README.md" "$work/"
cd "$work"
git add index.json README.md
if git diff --cached --quiet; then
echo "index.json is unchanged."
exit 0
fi
git -c user.name="github-actions[bot]" \
-c user.email="41898282+github-actions[bot]@users.noreply.github.com" \
commit -q -m "Index for ${GITHUB_SHA::7}"
git push -q origin "HEAD:$INDEX_BRANCH"
echo "Published index.json for ${GITHUB_SHA}." >> "$GITHUB_STEP_SUMMARY"
13 changes: 13 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,19 @@ maintainer too.
Every change to the `.lua` file needs a higher `Version`. A change to only the
README or example files doesn't — but if in doubt, bump it.

Add a line for the new version to the `## Changes` section of the README,
starting with the version:

```markdown
## Changes

- 1.1 — lists the last 10 runs instead of 5
- 1.0 — first version
```

OpenSAK's Macro manager shows these lines when it offers the update (see
[The macro index](MAINTAINERS.md#the-macro-index)).

## License

By opening a pull request you agree that your contribution is published under
Expand Down
43 changes: 43 additions & 0 deletions MAINTAINERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,47 @@ The checker also uses two internal helpers of the parser (`_header_lines`,
`_KEY_LINE`) to see which keys are actually written. If "Test checker"
fails after a bump, look there first.

## The macro index

After every push to `main`, **Index** (`.github/workflows/index.yml`) runs
`scripts/build_index.py` and publishes `index.json` on the `index` branch:

https://raw.githubusercontent.com/OpenSAK-Org/OpenSAK-Macros/index/index.json

OpenSAK's Macro manager reads it to list, install and update macros, so it
doesn't have to walk the repository through the GitHub API. Everything in it
comes from `main`'s git history, so the same commit always gives the same
file; a push that changes nothing in it (e.g. to MAINTAINERS.md) leaves the
`index` branch alone. Run it by hand from the Actions tab ("Run workflow")
after changing the script or `.github/opensak-ref`.

Locally (with the header parser in `.tools/`, see `tests/`):

python scripts/build_index.py --metadata .tools/macro_metadata.py --ref main

Format (`schema` 1; fields are only ever added, a breaking change raises
`schema`):

| Field | |
|---|---|
| `schema`, `commit`, `generated` | format version; the `main` commit indexed and its date (UTC) |
| `macros[]` | one entry per folder under `macros/` that has `<folder>/<folder>.lua` |
| `.id`, `.folder`, `.file`, `.readme`, `.page` | folder name, paths in the repository, link to the folder on GitHub |
| `.name`, `.author`, `.version`, `.description`, `.requires_api`, `.permissions`, `.network`, `.headless` | the header, read with OpenSAK's parser (`headless` is null with a parser from before API 4) |
| `.files[]` | every file in the folder: `path`, `size`, `sha256`, and a download `url` pinned to `commit` |
| `.versions[]` | every version that was on `main`, newest first: `version`, `published` (date it first reached `main`), `commit`, `requires_api`, `permissions`, `network`, and `size`, `sha256`, `url` of the macro file |
| `.published`, `.updated` | when the first and the current version reached `main` |
| `.changelog[]` | `version` and `text` from the `## Changes` list in the README, newest first |
| `.error` | only when the header couldn't be read |
| `withdrawn[]` | macros that were on `main` and were removed: `id`, `folder`, `name`, `last_version`, `withdrawn` (date), and `versions[]` with the `sha256` of each, so OpenSAK can recognise an installed copy |

Hashes are SHA-256 of the bytes in git, which are the bytes the `url`s
serve. A macro is identified by its folder name, so renaming a folder lists
the old name as withdrawn and the new one as a new macro.

The `index` branch must stay writable by GitHub Actions: don't put it under
the "Protect main" ruleset or any other ruleset that blocks pushes.

## Repository settings this relies on

- Ruleset "Protect main" on the default branch: pull request required (0
Expand All @@ -80,3 +121,5 @@ fails after a bump, look there first.
- Settings → General → Pull Requests → **Allow auto-merge** on.
- Labels `macro problem`, `auto-merge` and `needs review`.
- Discussions on.
- The `index` branch is created by the first **Index** run and left
unprotected (see [The macro index](#the-macro-index)).
Loading
Loading