Skip to content

Fix Pre-generate Contributor Data - #1803

Open
David Aniebo (Webmekanic) wants to merge 3 commits into
microsoft:mainfrom
Webmekanic:fix-pre-generate-contributor-data
Open

David Aniebo (Webmekanic) wants to merge 3 commits into
microsoft:mainfrom
Webmekanic:fix-pre-generate-contributor-data

Conversation

@Webmekanic

Copy link
Copy Markdown
Contributor

Summary

The Contributors page called https://api.github.com/repos/{repo}/contributors while it was being rendered, once per repo for each language version of the page. Our internal Azure DevOps build (Aspire.Dev-Build) runs these calls without a token. On shared agent IPs, they hit GitHub's anonymous rate limit and return 403 Forbidden: builds 3087220 and 3086870 logged 41–42 of these errors. When a call failed, the page was built and deployed without that repo's contributor list, in every language version.

The build shouldn't depend on an outside service at all. These calls can fail, they make the output vary between builds, and they slow down CI. Setting a PUBLIC_GITHUB_TOKEN would still leave network calls in the build. Because it's a PUBLIC_* variable, the token could also end up in client code.

Validation

pnpm exec vitest run --config vitest.config.ts tests/unit/contributor-list.vitest.test.ts passes (9 tests). It covers:

  • ContributorList renders from the committed JSON, and a stubbed fetch is never called
  • the ignore list and case-insensitive repo names
  • a repo with no data fails the build with a message to run pnpm update:contributors
  • every configured repo has a non-empty, well-formed list in github-contributors.json
  • the script follows pagination and sends the token
  • the script refuses to save a partial list after a 403, and rejects next-page links that point elsewhere
  • the guard finds no api.github.com in build-time code
  • tests/unit/release-contributors.vitest.test.ts passes.
  • pnpm exec eslint --max-warnings 0 on the changed .ts/.mjs files is clean.
  • pnpm update:contributors ran successfully against GitHub and generated the committed JSON: 306 / 40 / 88 / 78 / 19 contributors. Pagination worked, since microsoft/aspire needed 4 pages.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The implementation removes build-time network access while providing validated, automated contributor-data refreshes.

Review effort: Balanced
Findings: None

What changed in this PR

Pre-generates contributor data so Astro builds no longer depend on GitHub API availability.

Changes:

  • Adds a paginated contributor-data updater and committed JSON snapshot.
  • Renders contributor lists from local data and removes runtime fetching/caching.
  • Integrates contributor refreshes into automation with validation tests.
File Description
.github/​instructions/​astro.instructions.md Documents contributor data updates.
.github/​workflows/​update-integration-data.yml Supplies authentication and stages generated data.
src/​frontend/​astro.config.mjs Removes the contributor cache integration.
src/​frontend/​package.json Adds contributor update commands.
src/​frontend/​scripts/​check-data-files.mjs Requires generated contributor data.
src/​frontend/​scripts/​update-contributors.ts Fetches, validates, and writes contributor data.
src/​frontend/​scripts/​update-integration-data.ps1 Includes contributor data in automation.
src/​frontend/​src/​components/​ContributorList.astro Renders committed contributor data.
src/​frontend/​src/​content/​i18n/​en.json Removes obsolete unavailable-state translations.
src/​frontend/​src/​data/​github-contributors.json Stores generated contributor records.
src/​frontend/​src/​utils/​contributors.ts Removes build-time GitHub fetching and caching.
src/​frontend/​tests/​unit/​contributor-list.vitest.test.ts Tests rendering, generation, and network guards.
src/​frontend/​tests/​unit/​contributors.vitest.test.ts Removes obsolete runtime-fetch tests.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

it('has no api.github.com requests outside scripts/', () => {
const offenders = roots
.flatMap((root) => [...sourceFiles(join(frontendRoot, root))])
.filter((file) => readFileSync(join(frontendRoot, file), 'utf8').includes('api.github.com'));
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants