Skip to content

Publish a macro index with versions, sizes and SHA-256 hashes - #21

Open
nagisml wants to merge 1 commit into
OpenSAK-Org:mainfrom
nagisml:feature/macro-index
Open

nagisml wants to merge 1 commit into
OpenSAK-Org:mainfrom
nagisml:feature/macro-index

Conversation

@nagisml

@nagisml nagisml commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

@AgreeDK this is my idea feel free to change it in any way

What

After every push to main, a new Index workflow builds index.json and publishes it on an index branch:

https://raw.githubusercontent.com/OpenSAK-Org/OpenSAK-Macros/index/index.json

OpenSAK's Macro manager can read this one file to browse, install and update macros. That is the planned "update from OpenSAK-Macros" follow-up to OpenSAK-Org/OpenSAK#938. The manager won't need the GitHub API or rate-limited directory listings.

What the index holds

For each macro:

  • Header: name, author, version, description, Requires-API, permissions, network, headless. It is read with OpenSAK's own parser from .github/opensak-ref, the same one the PR check uses.
  • Files: every file in the folder, with size, sha256 and a download url pinned to the commit. The bytes behind a URL never change.
  • Versions: every version that was on main, newest first, with publish date, sha256, size, Requires-API and permissions. The manager can use them to see which version is installed and whether an update needs a newer OpenSAK or asks for more permissions.
  • Changelog: the ## Changes list from the README (- 1.1 — text). All current macros already use this format.

Macros removed from main are listed under withdrawn, with the hashes of all their versions, so OpenSAK can recognise an installed copy.

MAINTAINERS.md ("The macro index") describes the full format. It is schema 1: new fields may be added later, and a breaking change would raise the schema number.

How

  • scripts/build_index.py only reads git objects (ls-tree, show, log --first-parent) and never runs macro code. The same commit always gives the same file.
  • .github/workflows/index.yml runs only on pushes to main and on manual dispatch. It publishes to the index branch, which holds only index.json and a short README. If the file is unchanged, nothing is pushed.
  • The script runs from main, so everything in the index has passed the pull request checks.
  • Hashes are taken over the bytes git stores, which are the bytes raw.githubusercontent.com serves.

Docs

  • MAINTAINERS.md: new "The macro index" section.
  • CONTRIBUTING.md and the template README: add a ## Changes line for each new version.

Tests

tests/test_build_index.py has 13 new tests. They cover header, files and hashes, CRLF bytes, version history and dates, a version republished without a bump, changelog parsing, withdrawn macros, a macro that comes back after removal, an unreadable header, identical output from the same commit, and a check against this repository itself. "Test checker" runs them.

Locally, all 54 tests pass with the pinned parser and with current OpenSAK beta.

After merging

  • The first Index run creates the index branch. No ruleset may block pushes to it; "Protect main" covers only the default branch.
  • The pinned parser predates the Headless key, so headless is null until .github/opensak-ref is moved to a commit that has it.

@github-actions github-actions Bot added the needs review Waits for a maintainer label Nov 18, 2026
Add scripts/build_index.py and the Index workflow. After every push to
main, the workflow builds index.json from main's git history and
publishes it on the "index" branch:

  https://raw.githubusercontent.com/OpenSAK-Org/OpenSAK-Macros/index/index.json

OpenSAK's Macro manager can read this file to list, install and update
macros without walking the repository through the GitHub API.

Per macro the index holds:
- the header, read with OpenSAK's own parser (from .github/opensak-ref)
- every file in its folder, with size, SHA-256 and a download URL
  pinned to the commit
- every version that was on main: publish date, SHA-256, size,
  Requires-API and permissions of the macro file
- the changelog from the "## Changes" list in its README

Macros removed from main are listed under "withdrawn" with the hashes
of all their versions, so an installed copy can be recognised.

The script only reads git objects and never runs macro code. The same
commit always gives the same file, and a run that changes nothing
pushes nothing. MAINTAINERS.md describes the format (schema 1).
CONTRIBUTING.md and the template README now ask for a "## Changes"
line per version.
@nagisml
nagisml force-pushed the feature/macro-index branch from a617aee to dcb2fab Compare November 18, 2026 20:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs review Waits for a maintainer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant