Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion .github/scripts/pr-suites-packs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,16 @@ if [[ -z "$PR_NUMBER" ]]; then
for pack_dir in "$LANGUAGE/src" "$LANGUAGE/lib" "$LANGUAGE/ext" "$LANGUAGE/ext-library-sources"; do
if [[ -f "$pack_dir/qlpack.yml" ]]; then
echo "[+] Compiling Pack: $pack_dir"
codeql pack install "$pack_dir"
# `ext`/`ext-library-sources` are CodeQL model/extension packs (`extensionTargets`,
# no `dependencies`) - deliberately skip `codeql pack install` for them. Installing
# writes a `codeql-pack.lock.yml` with an empty `dependencies: {}` map, and a
# checked-in lock file in that state makes `codeql pack create` emit a bogus
# `addsTo.pack '...' is not an extension target of '...'` warning for every data
# extension in the pack (a known CodeQL CLI bug, see
# https://github.com/github/codeql/issues/20211). See CONTRIBUTING.md.
if [[ "$pack_dir" != "$LANGUAGE/ext" && "$pack_dir" != "$LANGUAGE/ext-library-sources" ]]; then
codeql pack install "$pack_dir"
fi
codeql pack create "$pack_dir"
fi
done
Expand Down
11 changes: 9 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,13 @@ jobs:
GITHUB_TOKEN: ${{ github.token }}
run: |
gh extension install github/gh-codeql
gh codeql pack install "${{ matrix.language }}/ext/"
# NOTE: deliberately no `gh codeql pack install` here. `ext` is a CodeQL model/extension
# pack (`library: true`, `extensionTargets`, no `dependencies`) - installing it just
# (re)writes a `codeql-pack.lock.yml` with an empty `dependencies: {}` map, and a
# checked-in lock file in that state makes a subsequent `codeql pack create` emit a
# bogus `addsTo.pack '...' is not an extension target of '...'` warning for every data
# extension in the pack (a known CodeQL CLI bug, see
# https://github.com/github/codeql/issues/20211). See CONTRIBUTING.md.
gh codeql pack create "${{ matrix.language }}/ext/"

library-sources:
Expand Down Expand Up @@ -221,7 +227,8 @@ jobs:
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
codeql pack install "${{ matrix.language }}/ext-library-sources/"
# NOTE: deliberately no `codeql pack install` here - see the matching comment in the
# `extensions` job above.
codeql pack create "${{ matrix.language }}/ext-library-sources/"

configs:
Expand Down
11 changes: 9 additions & 2 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -226,7 +226,13 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
echo "Publishing codeql-${{ matrix.language }}-extensions."
codeql pack install "${{ matrix.language }}/ext"
# NOTE: deliberately no `codeql pack install` here. `ext` is a CodeQL model/extension
# pack (`library: true`, `extensionTargets`, no `dependencies`) - installing it just
# (re)writes a `codeql-pack.lock.yml` with an empty `dependencies: {}` map, and a
# checked-in lock file in that state makes a subsequent `codeql pack publish`/`create`
# emit a bogus `addsTo.pack '...' is not an extension target of '...'` warning for every
# data extension in the pack (a known CodeQL CLI bug, see
# https://github.com/github/codeql/issues/20211). See CONTRIBUTING.md.
codeql pack publish "${{ matrix.language }}/ext"

- name: Record publish result
Expand Down Expand Up @@ -307,7 +313,8 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
echo "Publishing codeql-${{ matrix.language }}-library-sources."
codeql pack install "${{ matrix.language }}/ext-library-sources"
# NOTE: deliberately no `codeql pack install` here - see the matching comment in the
# `extensions` job above.
codeql pack publish "${{ matrix.language }}/ext-library-sources"

- name: Record publish result
Expand Down
17 changes: 14 additions & 3 deletions .github/workflows/update-codeql-version.yml
Original file line number Diff line number Diff line change
Expand Up @@ -91,8 +91,10 @@ jobs:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -euo pipefail
# Exclusions (kept in sync with the same exclusions in
# .github/scripts/pin-codeql-library-versions.sh):
# Exclusions (the first three below are also excluded in
# .github/scripts/pin-codeql-library-versions.sh; the last one is
# NOT - ext/ext-library-sources still need their extensionTargets
# versions pinned, but must be skipped here to avoid lock-file churn):
# - ./ql/hotspots is a standalone local dev tool (see
# ql/hotspots/README.md and .github/workflows/hotspots.yml), not
# one of the per-language src/lib/ext/ext-library-sources packs
Expand All @@ -106,7 +108,16 @@ jobs:
# - ./codeql and */.codeql are a locally-cloned github/codeql
# checkout (used by ql/hotspots) and CodeQL's own per-pack build
# caches, respectively - neither is a repo pack either.
for dir in $(find . -name qlpack.yml -not -path "./ql/hotspots/*" -not -path "./codeql_home/*" -not -path "./codeql/*" -not -path "*/.codeql/*" -exec dirname {} \;); do
# - <lang>/ext and <lang>/ext-library-sources are CodeQL model/extension packs
# (`extensionTargets`, no `dependencies`) - NOT excluded from
# pin-codeql-library-versions.sh (their extensionTargets versions still need
# pinning), but excluded here: `codeql pack upgrade` (re)writes their
# codeql-pack.lock.yml with an empty `dependencies: {}` map, and a checked-in
# lock file in that state makes a later `codeql pack create`/`publish` emit a
# bogus `addsTo.pack '...' is not an extension target of '...'` warning for
# every data extension in the pack (a known CodeQL CLI bug, see
# https://github.com/github/codeql/issues/20211). See CONTRIBUTING.md.
for dir in $(find . -name qlpack.yml -not -path "./ql/hotspots/*" -not -path "./codeql_home/*" -not -path "./codeql/*" -not -path "*/.codeql/*" -not -path "*/ext/*" -not -path "*/ext-library-sources/*" -exec dirname {} \;); do
echo "::group::codeql pack upgrade $dir"
codeql pack upgrade "$dir"
echo "::endgroup::"
Expand Down
21 changes: 21 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -293,6 +293,27 @@ coding agent) in the loop for the hard part — fixing whatever the new CLI brea
> `codeql pack upgrade`/the pinning script can never resolve it. This is a pre-existing, unrelated
> quirk of that tool, not something the version-bump automation needs to (or can) fix.

> [!NOTE]
> <a id="ext-packs-no-install"></a>**Why `<language>/ext` and `<language>/ext-library-sources` are
> never `codeql pack install`ed or `codeql pack upgrade`d:** these are CodeQL
> [model/extension packs](https://docs.github.com/en/code-security/tutorials/customize-code-scanning/create-and-work-with-codeql-packs#creating-a-codeql-model-pack)
> — `library: true`, an `extensionTargets` map, and (by design) **no `dependencies`**. `codeql pack
> install`/`codeql pack upgrade` still (re)write a `codeql-pack.lock.yml` for them, but since there's
> nothing to resolve it's always an empty `dependencies: {}` map. A *checked-in* lock file in that
> state triggers a known CodeQL CLI bug
> ([github/codeql#20211](https://github.com/github/codeql/issues/20211)): a later `codeql pack
> create`/`codeql pack publish` on the same pack emits a bogus `WARNING: In extension for
> codeql/<language>-all:<extensible>, addsTo.pack 'codeql/<language>-all' is not an extension target
> of '...'` for every data extension file in the pack, even though `extensionTargets` correctly lists
> that pack. CI (`ci.yml`'s `extensions`/`library-sources` jobs, `publish.yml`'s `extensions`/
> `library_sources_extensions` jobs, `pr-suites-packs.sh`) and
> [`update-codeql-version.yml`](#updating-the-pinned-codeql-clilibrary-version)'s `codeql pack
> upgrade` loop deliberately skip `install`/`upgrade` for these two pack types and go straight to
> `codeql pack create`/`publish` — and no `codeql-pack.lock.yml` should ever be committed for them.
> If you run `codeql pack install`/`upgrade` against one of these directories locally while
> developing (e.g. to sanity-check a new data extension), delete the resulting
> `codeql-pack.lock.yml` before committing.

> [!WARNING]
> The `.codeqlversion` bump and the pack version bumps don't have to land in the same PR, but
> splitting them is risky: [#124][pr-124] refreshed `.codeqlversion` and every language's
Expand Down
4 changes: 0 additions & 4 deletions csharp/ext-library-sources/codeql-pack.lock.yml

This file was deleted.

4 changes: 0 additions & 4 deletions csharp/ext/codeql-pack.lock.yml

This file was deleted.

4 changes: 0 additions & 4 deletions go/ext/codeql-pack.lock.yml

This file was deleted.

4 changes: 0 additions & 4 deletions java/ext-library-sources/codeql-pack.lock.yml

This file was deleted.

4 changes: 0 additions & 4 deletions java/ext/codeql-pack.lock.yml

This file was deleted.

4 changes: 0 additions & 4 deletions python/ext/codeql-pack.lock.yml

This file was deleted.

Loading