telemetry: close privacy-review gaps in exegraph + hooks; backfill audit docs#8600
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR closes telemetry privacy-review gaps by hashing previously-raw user-derived identifiers (exegraph.step.name, exegraph.step.deps, and most hooks.name values) and backfills/realigns the metrics-audit documentation with the telemetry fields/events actually emitted by cli/azd/.
Changes:
- Hash execution-graph step names/dependencies and hash hook names by default (raw only for an allowlisted set of built-in lifecycle hook names).
- Centralize the lifecycle hook allowlist so both
azd hooks runand the hooks runner use the same source of truth. - Backfill and correct the metrics-audit documentation (schema, feature matrix, hashing checklist) and update contributor guidance for telemetry changes.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
.vscode/cspell.misc.yaml |
Adds audit/telemetry terms to the cspell misc allowlist. |
cli/azd/cmd/hooks.go |
Switches to the shared hook-name allowlist for raw vs hashed telemetry emission. |
cli/azd/pkg/ext/hooks_runner.go |
Hashes hooks.name by default and emits raw only for allowlisted lifecycle hook names. |
cli/azd/pkg/ext/known_hooks.go |
Introduces a shared allowlist of built-in lifecycle hook names. |
cli/azd/pkg/exegraph/scheduler.go |
Hashes exegraph.step.name and exegraph.step.deps attributes. |
cli/azd/internal/tracing/fields/fields.go |
Updates field comments to reflect hashing requirements and tag safety rationale. |
cli/azd/AGENTS.md |
Updates contributor guidance to require telemetry doc + coverage-test updates when telemetry changes. |
docs/specs/metrics-audit/feature-telemetry-matrix.md |
Expands/repairs command ↔ telemetry inventory and cross-cutting subsystem coverage. |
docs/specs/metrics-audit/privacy-review-checklist.md |
Splits always-hashed vs conditional-hash fields and documents hashing requirements more precisely. |
docs/specs/metrics-audit/telemetry-schema.md |
Backfills/updates schema for events/fields and documents hashing/PII-risk notes. |
hemarina
added
area/telemetry
Collaborator
Azure Dev CLI Install InstructionsInstall scriptsMacOS/Linux
bash: pwsh: WindowsPowerShell install MSI install Standalone Binary
MSI
Documentationlearn.microsoft.com documentationtitle: Azure Developer CLI reference
|
JeffreyCA
approved these changes
Jun 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix https://github.com/Azure/azure-dev-pr/issues/1793, part of https://github.com/Azure/azure-dev-pr/issues/1772
Summary
Closes telemetry privacy-review gaps surfaced by an audit of the metrics-audit docs in
docs/specs/metrics-audit/against the actual telemetry-emitting code undercli/azd/.The audit found:
exegraph.step.nameandexegraph.step.depswere emitted RAW even though step names embed user-definedazure.yamlservice/layer names (e.g.,deploy-<svc.Name>,provision-<layer.Name>).hooks.namewas emitted RAW unconditionally frompkg/ext/hooks_runner.go; only the (lower-traffic)cmd/hooks.gopath used an allowlist.internal/tracing/fields/fields.go,internal/tracing/events/events.go) were undocumented in the privacy-review materials. Several existing rows also had inaccurate field names and enum descriptions.Privacy fixes (code changes)
e2d6080pkg/exegraph/scheduler.go: switchExeGraphStepNameKey/ExeGraphStepDepsKeytofields.StringHashed/fields.StringSliceHashed. Step names embedazure.yamlservice/layer names (verified atinternal/cmd/service_graph.go:356-358,up_graph.go:770,provision_graph.go:520-522).Tagsleft raw — verified all&exegraph.Step{Tags: ...}constructors use hardcoded string literals only.74cf84epkg/ext/hooks_runner.gonow hasheshooks.nameby default and only emits raw when the name is in a shared allowlist of built-in lifecycle hooks. Extracted the allowlist fromcmd/hooks.gointo a newpkg/ext/known_hooks.go(KnownHookNames) so both emit sites stay in sync.Documentation backfill
380d822telemetry-schema.md— documented all 136 attribute keys infields.goand all 25 event constants inevents.go. Added Hashing section and PII-Risk Assessment for newly-documented fields.cf8f417feature-telemetry-matrix.md— added Tool Management, Copilot (Agent), and Cross-Cutting Subsystems sections; corrected ✅/❌ flags on commands now known to emit additional telemetry (e.g.,provision,deploy,package,extension).5d59b84privacy-review-checklist.md— split the hashed-fields list into "Fields That Must Be Hashed" (always-hashed) and "Fields With Conditional Hashing" (subscription.id,pack.builder.image/tag) to remove a previously-misleading "always hashed" claim.1ff3773hooks.kindenum description (it's the script runtimesh/pwsh/js/ts/python/dotnet, not pre/post), replaced fictionalextension.source.id/.type/.dependency.*field names with the real keys, removed falseextension sourcesubcommand attribution, and closed thetool.install.strategyenum gap.Verification
Every documented field/event/enum value was verified at a concrete code location, not against comments or documentation. Spot examples:
events.ArmDeploySubscriptionEvent→pkg/azapi/standard_deployments.go:228;events.CopilotInitializeEvent→internal/agent/copilot_agent.go:81;events.AksPostprovisionSkipEvent→pkg/project/service_target_aks.go:990.validation.preflight.outcomevalues verified against constants atpkg/infra/provisioning/bicep/bicep_provider.go:2549-2554;provision.cancellationvalues verified atpkg/infra/provisioning/bicep/interrupt.go;hooks.kindvalues verified atpkg/tools/language/executor.go:14-31.subscription.idverified atpkg/environment/local_file_data_store.go:198-202andstorage_blob_data_store.go:186-190(UUID-valid → raw, otherwise hashed);pack.builder.image/tagverified atpkg/project/container_helper.go:1126-1137(userDefinedImage == true→ hashed).hooks.namepost-fix — both emit sites (cmd/hooks.go:149-152andpkg/ext/hooks_runner.go:180-183) now use the sharedext.KnownHookNamesallowlist; grep confirms no third raw emit site exists.Files changed
Backward compatibility
fields.StringHashed/fields.StringSliceHashed) — the existing helpers used throughout the codebase for the other 4 always-hashed fields (project.template.id,project.template.version,project.name,env.name).Testing
pkg/ext/,pkg/exegraph/,cmd/middleware/continue to pass.Test_Lazy_AzdContext_Resolution/Test_Lazy_Project_Config_Resolutionfailures on Windows are unrelated (caused by a strayC:\Users\<user>\azure.yamlfrom prior test runs walking up to the user-home root; reproduces on baselinemain).