Repository navigation
fix: double-counted metrics and SCIM PATCH replace - #790
Merged
Merged
Conversation
FastAPI 0.142 records its own request histogram into the global MeterProvider, which on the engine is scalo's. So /metrics carried http_server_request_duration_seconds twice, once from scalo's middleware and once under otel_scope_name="fastapi", and the app status query counted every engine request twice. The engine API now turns FastAPI's request metrics off. The KEDA shim keeps them, it has no scalo histogram.
The schema phase built a fresh scalo MetricsManager on every pass. Under the OpenTelemetry backend that is a new MeterProvider the SDK refuses to install ("Overriding of current MeterProvider is not allowed") and then leaves running beside the real one. The daemon now registers the schema gauges on the manager it already serves on /metrics, under the same dfe_schema_* names. `dfe schema apply` still builds its own, once per run.
A SCIM group PATCH replace only ever added members. It now swaps the whole set, per RFC 7644 section 3.5.2.3. A filtered replace swaps just the member it names, and a filter that matches no member is a 400 noTarget. A path-less replace or add carrying members is honoured too. The new member list is worked out first and written once, so the audit event names only who joined and who left, and a PATCH that changes nothing emits nothing.
|
Released in v1.22.18 -- https://github.com/hyperi-io/dfe-engine/releases/tag/v1.22.18 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three follow-ups to #789.
http.server.request.durationinto whatever MeterProvider is global, and on the engine that is scalo's. So/metricscarriedhttp_server_request_duration_secondstwice, the second family underotel_scope_name="fastapi", and the app status request query counted every engine request twice. The engine API now passes{**FASTAPI_TELEMETRY, "metrics": False}. The KEDA shim is unchanged, it has no scalo histogram to collide with.create_metrics()on every pass, so the daemon built a second MetricsManager and MeterProvider beside ServiceApp's, loggedOverriding of current MeterProvider is not allowed, and left the orphan's reader running.create_appnow buildsSchemaMetricsonce on the manager the daemon serves, and the lifespan hands it tobootstrap_clickhouse. The gauges keep theirdfe_schema_*names whatever namespace the manager carries.dfe schema applybuilds its own manager, once per run.replaceonmembersonly ever added. It now swaps the whole set (RFC 7644 section 3.5.2.3), amembers[value eq "x"]replace swaps that one member, and a filter matching no member answers 400noTarget. A path-less add or replace carryingmembersis honoured, and one carrying none leaves them alone. The member list is worked out before anything is written and stored in one update, so the audit event'saddedandremovedare exactly who changed.test_daemon_metrics.pybuilds the daemon's shape in a child process, because the global MeterProvider can only be set once, and checks for one histogram family, one MeterProvider, no override refusal and thedfe_schema_*names.test_schema_metrics.pycovers both manager namespaces.test_scim_change_audit.pycovers each replace form, the noTarget refusal and the no-change cases. Each fails with its fix reverted.Done when CI is green on every job.