Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,16 @@ the compatibility and migration notes before upgrading.
(`subject_assurance: unknown`) and license fields are preserved verbatim.
Migration 033 adds the `supply_chain_*` tables; with the module disabled
nothing else changes. See ADR-0017 and `docs/execplans/supply-chain.md`.
- Exact-version license evidence for npm, NuGet, and Maven components from
explicitly configured registry routes (`GRAPHNEST_SUPPLY_CHAIN_REGISTRY_*`),
parsed with a bounded SPDX 2.3 expression parser against the pinned SPDX
License List 3.27.0. Evidence rows are immutable and carry raw values,
parse status, resolver and list versions, content hashes, and outcomes;
per-occurrence assessments report resolved, declared, conflict, unlicensed,
or unknown and are shown in the component table and a new evidence detail
view (`GET /v1/supply-chain/repositories/{id}/component`). No route means no
outbound license traffic. Migration 034 adds the evidence, enrichment-job,
and assessment tables.

## [0.5.0] - 2026-09-18

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,7 @@ With `GRAPHNEST_SUPPLY_CHAIN=true` in durable mode, `graphnest-server` collects
What the inventory is and is not:

- A GitHub dependency-graph export is a **timestamped observation of the default branch**. The endpoint has no ref selector, so snapshots report `subject_assurance: unknown`; GraphNest never copies the indexed or current HEAD into a snapshot.
- GitHub Enterprise Server does not populate dependency license fields; `license_declared_raw`/`license_concluded_raw` are preserved verbatim (typically `NOASSERTION`) and are never mapped to a license. Exact-version license evidence is a separate, later enrichment layer.
- GitHub Enterprise Server does not populate dependency license fields; `license_declared_raw`/`license_concluded_raw` are preserved verbatim (typically `NOASSERTION`) and are never mapped to a license. Exact-version license evidence comes only from registry routes you configure (`GRAPHNEST_SUPPLY_CHAIN_REGISTRY_{NPM,NUGET,MAVEN}_URL` and companion secret-file settings); without a route no license traffic is produced, and a private route never falls back to a public registry. SPDX expressions are parsed against the pinned SPDX License List 3.27.0 with AND/OR/WITH structure preserved; `NOASSERTION`, `NONE`, `UNLICENSED`, unknown identifiers, license files, and URLs stay what they are.
- Components without a purl or version stay visible. Dependency scope (`root`/`direct`/`transitive`) is derived only from resolved `DEPENDS_ON` edges leaving a described root; a flattened list yields `unknown`, never `direct`.
- A failed refresh (403, 404, rate limit, malformed or oversized document, outage) records a collection attempt and leaves the last successful snapshot in place; the status reports `collection: failed` alongside the retained inventory.
- Inventory eligibility is repository authorization alone. It works for repositories with no Zoekt index, no SCIP upload, and no graph enrichment, and inventory work never blocks lexical indexing.
Expand Down
39 changes: 35 additions & 4 deletions cmd/graphnest-server/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ import (
"github.com/balcsida/graphnest/internal/sso/githuboauth"
"github.com/balcsida/graphnest/internal/sso/oidc"
"github.com/balcsida/graphnest/internal/supplychain"
"github.com/balcsida/graphnest/internal/supplychain/license"
"github.com/balcsida/graphnest/internal/webhook"
"github.com/balcsida/graphnest/internal/webui"
"github.com/balcsida/graphnest/internal/zoekt"
Expand Down Expand Up @@ -451,8 +452,23 @@ func newDurableRuntime(ctx context.Context, settings config.Config, logger *slog
var extras []func(*http.ServeMux)
var supplyChainDone []<-chan struct{}
if settings.SupplyChain.Enabled {
supplyChainService := &supplychain.Service{Store: store, Authorizer: authz.NewPostgres(store), Interval: settings.SupplyChain.Interval, MaxResults: settings.Limits.MaxResults}
supplyChainDone = startSupplyChain(loopCtx, settings.SupplyChain, store, githubClient, metrics, logger)
routes, err := license.RoutesFromEnv(os.Getenv, license.ReadSecretFile)
if err != nil {
cancel()
<-done
<-reconcileDone
return fail(fmt.Errorf("supply chain registry routes: %w", err))
}
registry, err := license.NewRegistry(routes)
if err != nil {
cancel()
<-done
<-reconcileDone
return fail(fmt.Errorf("supply chain registry routes: %w", err))
}
supplyChainService := &supplychain.Service{Store: store, Authorizer: authz.NewPostgres(store), Interval: settings.SupplyChain.Interval, MaxResults: settings.Limits.MaxResults,
License: store, EnrichmentEcosystems: registry.Ecosystems()}
supplyChainDone = startSupplyChain(loopCtx, settings.SupplyChain, store, githubClient, registry, metrics, logger)
extras = append(extras, func(mux *http.ServeMux) {
httpapi.RegisterSupplyChain(mux, auth.requestAuth, supplyChainService, settings.Limits.MaxResults, settings.Limits.MaxResponseBytes)
})
Expand All @@ -478,8 +494,21 @@ func newDurableRuntime(ctx context.Context, settings config.Config, logger *slog
// startSupplyChain runs the inventory scheduler and collection workers inside
// the server process. They share nothing with the indexer, so inventory work
// can neither block nor be blocked by lexical indexing (ADR-0017).
func startSupplyChain(ctx context.Context, settings config.SupplyChain, store *postgres.Store, client *githubapp.Client, metrics *observability.Metrics, logger *slog.Logger) []<-chan struct{} {
func startSupplyChain(ctx context.Context, settings config.SupplyChain, store *postgres.Store, client *githubapp.Client, registry *license.Registry, metrics *observability.Metrics, logger *slog.Logger) []<-chan struct{} {
var done []<-chan struct{}
hostname, _ := os.Hostname()
var enricher *license.Worker
if len(registry.Ecosystems()) > 0 {
enricher = &license.Worker{Store: store, Registry: registry, Owner: fmt.Sprintf("%s-%d-enrich", hostname, os.Getpid()), Logger: logger, Observer: metrics}
enrichDone := make(chan struct{})
done = append(done, enrichDone)
go func() {
defer close(enrichDone)
if err := enricher.Run(ctx); err != nil && ctx.Err() == nil {
logger.Error("supply chain enrichment worker stopped", "error", err)
}
}()
}
scheduler := &supplychain.Scheduler{Store: store, Interval: settings.Interval}
schedulerDone := make(chan struct{})
done = append(done, schedulerDone)
Expand All @@ -502,12 +531,14 @@ func startSupplyChain(ctx context.Context, settings config.SupplyChain, store *p
}
}
}()
hostname, _ := os.Hostname()
for worker := range settings.Workers {
collector := &supplychain.Collector{
Store: store, GitHub: client, Owner: fmt.Sprintf("%s-%d-%d", hostname, os.Getpid(), worker), MaxDocumentBytes: settings.MaxDocumentBytes,
Limits: supplychain.Limits{MaxComponents: settings.MaxComponents}, Logger: logger, Observer: metrics,
}
if enricher != nil {
collector.Enricher = enricher
}
workerDone := make(chan struct{})
done = append(done, workerDone)
go func() {
Expand Down
10 changes: 10 additions & 0 deletions deploy/helm/graphnest/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,16 @@ the only outbound call is the configured GitHub API endpoint. Snapshots live in
PostgreSQL (`supply_chain_*` tables, created by the normal migration Job); see
the repository operations guide for lifecycle, recovery, and metrics.

License enrichment is off until a registry route is set under
`server.supplyChain.registries.{npm,nuget,maven}.url` (HTTPS). Optional
`namespaces` restrict what the route may answer for, `allowPrivate` permits an
internal mirror on a private address, and `token`/`basic` mount the matching
key of the existing Secret named by `secrets.supplyChainRegistries` at
`/var/run/secrets/graphnest/registries/` (npm and NuGet: bearer token; Maven:
`user:password`); `registries.ca: true` mounts its `caKey` as the route CA.
Credentials never render into a ConfigMap. A private route is never bypassed
toward a public registry.

`breakGlass.enabled=true` exposes only the disabled-by-default local recovery
routes. It provisions no user name, password, hash, salt, or Secret and never
activates because OIDC is unavailable. Provision and rotate the operator
Expand Down
13 changes: 12 additions & 1 deletion deploy/helm/graphnest/ci/optional-values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,20 @@ secrets:
githubOAuth: {name: graphnest-github-oauth, clientSecretKey: client-secret}
oidcCA: {name: graphnest-oidc-ca, key: ca.crt}
scim: {name: graphnest-scim, tokenKey: token}
supplyChainRegistries: {name: graphnest-registries, npmTokenKey: npm-token, nugetTokenKey: nuget-token, mavenBasicKey: maven-basic, caKey: ca.crt}
server:
scim: {enabled: true}
supplyChain: {enabled: true, interval: 12h, workers: 2, maxDocumentBytes: 33554432, maxComponents: 60000}
supplyChain:
enabled: true
interval: 12h
workers: 2
maxDocumentBytes: 33554432
maxComponents: 60000
registries:
npm: {url: https://npm.example.invalid/, namespaces: ["@acme", "@internal"], allowPrivate: false, token: true}
nuget: {url: "", namespaces: [], allowPrivate: false}
maven: {url: https://maven.example.invalid/repository/public/, namespaces: [com.acme], allowPrivate: true, basic: true}
ca: true
sso:
publicURL: https://graphnest.example.invalid
oidc: {enabled: true, issuerURL: https://id.example.invalid, clientID: graphnest, scopes: [openid, profile, email], linkClaim: sub, displayNameClaim: name}
Expand Down
21 changes: 21 additions & 0 deletions deploy/helm/graphnest/templates/configmaps.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,27 @@ data:
GRAPHNEST_SUPPLY_CHAIN_WORKERS: {{ .Values.server.supplyChain.workers | quote }}
GRAPHNEST_SUPPLY_CHAIN_MAX_DOCUMENT_BYTES: {{ printf "%d" (int64 .Values.server.supplyChain.maxDocumentBytes) | quote }}
GRAPHNEST_SUPPLY_CHAIN_MAX_COMPONENTS: {{ .Values.server.supplyChain.maxComponents | quote }}
{{- range $ecosystem, $route := (pick .Values.server.supplyChain.registries "npm" "nuget" "maven") }}
{{- if $route.url }}
{{- $prefix := printf "GRAPHNEST_SUPPLY_CHAIN_REGISTRY_%s_" (upper $ecosystem) }}
{{ $prefix }}URL: {{ $route.url | quote }}
{{- if $route.namespaces }}
{{ $prefix }}NAMESPACES: {{ join "," $route.namespaces | quote }}
{{- end }}
{{- if $route.allowPrivate }}
{{ $prefix }}ALLOW_PRIVATE: "true"
{{- end }}
{{- if $route.token }}
{{ $prefix }}TOKEN_FILE: /var/run/secrets/graphnest/registries/{{ $ecosystem }}-token
{{- end }}
{{- if $route.basic }}
{{ $prefix }}BASIC_FILE: /var/run/secrets/graphnest/registries/{{ $ecosystem }}-basic
{{- end }}
{{- if $.Values.server.supplyChain.registries.ca }}
{{ $prefix }}CA_FILE: /var/run/secrets/graphnest/registries/ca.crt
{{- end }}
{{- end }}
{{- end }}
{{- end }}
---
apiVersion: v1
Expand Down
22 changes: 22 additions & 0 deletions deploy/helm/graphnest/templates/server.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,9 @@ spec:
{{- if .Values.server.scim.enabled }}
- {name: scim-token, mountPath: /var/run/secrets/graphnest/scim/token, subPath: token, readOnly: true}
{{- end }}
{{- if and .Values.server.supplyChain.enabled .Values.secrets.supplyChainRegistries.name }}
- {name: supply-chain-registries, mountPath: /var/run/secrets/graphnest/registries, readOnly: true}
{{- end }}
{{- if .Values.server.sso.oidc.enabled }}
- {name: oidc-client-secret, mountPath: /var/run/secrets/graphnest/oidc/client-secret, subPath: client-secret, readOnly: true}
{{- if .Values.secrets.oidcCA.name }}
Expand Down Expand Up @@ -151,6 +154,25 @@ spec:
items:
- {key: {{ .Values.secrets.scim.tokenKey }}, path: token}
{{- end }}
{{- if and .Values.server.supplyChain.enabled .Values.secrets.supplyChainRegistries.name }}
- name: supply-chain-registries
secret:
secretName: {{ .Values.secrets.supplyChainRegistries.name }}
optional: false
items:
{{- if .Values.server.supplyChain.registries.npm.token }}
- {key: {{ .Values.secrets.supplyChainRegistries.npmTokenKey }}, path: npm-token}
{{- end }}
{{- if .Values.server.supplyChain.registries.nuget.token }}
- {key: {{ .Values.secrets.supplyChainRegistries.nugetTokenKey }}, path: nuget-token}
{{- end }}
{{- if .Values.server.supplyChain.registries.maven.basic }}
- {key: {{ .Values.secrets.supplyChainRegistries.mavenBasicKey }}, path: maven-basic}
{{- end }}
{{- if .Values.server.supplyChain.registries.ca }}
- {key: {{ .Values.secrets.supplyChainRegistries.caKey }}, path: ca.crt}
{{- end }}
{{- end }}
---
apiVersion: v1
kind: Service
Expand Down
11 changes: 11 additions & 0 deletions deploy/helm/graphnest/tests/render.sh
Original file line number Diff line number Diff line change
Expand Up @@ -384,6 +384,17 @@ require 'GRAPHNEST_SUPPLY_CHAIN_INTERVAL: "12h"' "$tmp/optional.yaml"
require 'GRAPHNEST_SUPPLY_CHAIN_WORKERS: "2"' "$tmp/optional.yaml"
require 'GRAPHNEST_SUPPLY_CHAIN_MAX_DOCUMENT_BYTES: "33554432"' "$tmp/optional.yaml"
require 'GRAPHNEST_SUPPLY_CHAIN_MAX_COMPONENTS: "60000"' "$tmp/optional.yaml"
require 'GRAPHNEST_SUPPLY_CHAIN_REGISTRY_NPM_URL: "https://npm.example.invalid/"' "$tmp/optional.yaml"
require 'GRAPHNEST_SUPPLY_CHAIN_REGISTRY_NPM_NAMESPACES: "@acme,@internal"' "$tmp/optional.yaml"
require 'GRAPHNEST_SUPPLY_CHAIN_REGISTRY_NPM_TOKEN_FILE: /var/run/secrets/graphnest/registries/npm-token' "$tmp/optional.yaml"
require 'GRAPHNEST_SUPPLY_CHAIN_REGISTRY_NPM_CA_FILE: /var/run/secrets/graphnest/registries/ca.crt' "$tmp/optional.yaml"
require 'GRAPHNEST_SUPPLY_CHAIN_REGISTRY_MAVEN_URL: "https://maven.example.invalid/repository/public/"' "$tmp/optional.yaml"
require 'GRAPHNEST_SUPPLY_CHAIN_REGISTRY_MAVEN_ALLOW_PRIVATE: "true"' "$tmp/optional.yaml"
require 'GRAPHNEST_SUPPLY_CHAIN_REGISTRY_MAVEN_BASIC_FILE: /var/run/secrets/graphnest/registries/maven-basic' "$tmp/optional.yaml"
reject 'GRAPHNEST_SUPPLY_CHAIN_REGISTRY_NUGET|GRAPHNEST_SUPPLY_CHAIN_REGISTRY_NPM_ALLOW_PRIVATE|GRAPHNEST_SUPPLY_CHAIN_REGISTRY_NPM_BASIC|nuget-token' "$tmp/optional.yaml"
require 'mountPath: /var/run/secrets/graphnest/registries' "$tmp/optional.yaml"
require 'secretName: graphnest-registries' "$tmp/optional.yaml"
reject 'supply-chain-registries|GRAPHNEST_SUPPLY_CHAIN_REGISTRY' "$tmp/minimal.yaml"
reject '^kind: Secret$|GRAPHNEST_SCIM_TOKEN: ' "$tmp/optional.yaml"
require 'GRAPHNEST_PUBLIC_URL: "https://graphnest.example.invalid"' "$tmp/scim.yaml"
require 'GRAPHNEST_SCIM_TOKEN_FILE: /var/run/secrets/graphnest/scim/token' "$tmp/scim.yaml"
Expand Down
42 changes: 39 additions & 3 deletions deploy/helm/graphnest/values.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,8 @@
"githubOAuth",
"mcpOAuth",
"oidcCA",
"scim"
"scim",
"supplyChainRegistries"
],
"properties": {
"runtime": {
Expand All @@ -81,6 +82,18 @@
},
"scim": {
"$ref": "#/definitions/scimSecret"
},
"supplyChainRegistries": {
"type": "object",
"additionalProperties": false,
"required": ["name", "npmTokenKey", "nugetTokenKey", "mavenBasicKey", "caKey"],
"properties": {
"name": {"$ref": "#/definitions/optionalKubernetesObjectName"},
"npmTokenKey": {"$ref": "#/definitions/secretKey"},
"nugetTokenKey": {"$ref": "#/definitions/secretKey"},
"mavenBasicKey": {"$ref": "#/definitions/secretKey"},
"caKey": {"$ref": "#/definitions/secretKey"}
}
}
}
},
Expand Down Expand Up @@ -266,13 +279,24 @@
"supplyChain": {
"type": "object",
"additionalProperties": false,
"required": ["enabled", "interval", "workers", "maxDocumentBytes", "maxComponents"],
"required": ["enabled", "interval", "workers", "maxDocumentBytes", "maxComponents", "registries"],
"properties": {
"enabled": {"type": "boolean"},
"interval": {"$ref": "#/definitions/duration"},
"workers": {"type": "integer", "minimum": 1, "maximum": 8},
"maxDocumentBytes": {"type": "integer", "minimum": 1, "maximum": 268435456},
"maxComponents": {"type": "integer", "minimum": 1, "maximum": 500000}
"maxComponents": {"type": "integer", "minimum": 1, "maximum": 500000},
"registries": {
"type": "object",
"additionalProperties": false,
"required": ["npm", "nuget", "maven", "ca"],
"properties": {
"npm": {"$ref": "#/definitions/registryRoute"},
"nuget": {"$ref": "#/definitions/registryRoute"},
"maven": {"$ref": "#/definitions/registryRoute"},
"ca": {"type": "boolean", "description": "Mount secrets.supplyChainRegistries[caKey] as the route CA for every configured registry"}
}
}
}
},
"resources": {
Expand Down Expand Up @@ -1030,6 +1054,18 @@
"type": "string"
}
},
"registryRoute": {
"type": "object",
"additionalProperties": false,
"required": ["url", "namespaces", "allowPrivate"],
"properties": {
"url": {"type": "string", "pattern": "^(|https://.+)$"},
"namespaces": {"type": "array", "items": {"type": "string", "minLength": 1}},
"allowPrivate": {"type": "boolean"},
"token": {"type": "boolean", "description": "Mount the ecosystem token key from secrets.supplyChainRegistries as a bearer token"},
"basic": {"type": "boolean", "description": "Mount the ecosystem basic key from secrets.supplyChainRegistries as user:password"}
}
},
"scimSecret": {
"type": "object",
"additionalProperties": false,
Expand Down
15 changes: 14 additions & 1 deletion deploy/helm/graphnest/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ secrets:
mcpOAuth: {name: "", keyKey: sealing-key}
oidcCA: {name: "", key: ca.crt}
scim: {name: "", tokenKey: token}
# Optional per-ecosystem registry credentials for license enrichment. Keys are mounted read-only; a missing key is simply not mounted.
supplyChainRegistries: {name: "", npmTokenKey: npm-token, nugetTokenKey: nuget-token, mavenBasicKey: maven-basic, caKey: ca.crt}
breakGlass: {enabled: false}
server:
replicas: 2
Expand Down Expand Up @@ -53,7 +55,18 @@ server:
mcpOAuth: {enabled: false}
scim: {enabled: false}
# Dependencies & Licenses inventory (ADR-0017). Disabled by default; enabling it needs no repository changes.
supplyChain: {enabled: false, interval: 24h, workers: 1, maxDocumentBytes: 16777216, maxComponents: 50000}
supplyChain:
enabled: false
interval: 24h
workers: 1
maxDocumentBytes: 16777216
maxComponents: 50000
# License enrichment routes; an ecosystem without a url produces no registry traffic.
registries:
npm: {url: "", namespaces: [], allowPrivate: false, token: false}
nuget: {url: "", namespaces: [], allowPrivate: false, token: false}
maven: {url: "", namespaces: [], allowPrivate: false, basic: false}
ca: false
resources: {requests: {cpu: 250m, memory: 256Mi}, limits: {cpu: "1", memory: 1Gi}}
nodeSelector: {}
affinity: {}
Expand Down
Binary file modified docs/images/supply-chain-dark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/supply-chain-light.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading