fix(ffe): make exposure event test deterministic - #7465
Conversation
|
|
cb38a8a to
7c845b7
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7c845b7d20
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| assert result["value"] == "on", f"Expected 'on', got {result['value']!r}" | ||
|
|
||
| # Exposure delivery is asynchronous; wait before the next setup replaces Remote Config. | ||
| wait_for_exposure_event({self.flag}, self.targeting_key) |
There was a problem hiding this comment.
Keep exposure assertions out of setup
When this file is xfailed by a manifest for unsupported tracer versions (for example the file-level version gates in manifests/dotnet.yml, manifests/golang.yml, and manifests/python.yml), this assertion now runs inside the repository's custom setup loop in pytest_collection_finish before pytest executes the xfailed test. A timeout here is re-raised as a setup/collection failure instead of being reported as an expected test failure, so running those declared-unsupported versions can abort the scenario. Keep the wait/assertion in the test body, or make the setup path avoid failing for xfailed items.
Useful? React with 👍 / 👎.
🎉 All green!🧪 All tests passed 🔗 Commit SHA: aa82f0d | Docs | Datadog PR Page | Give us feedback! |
Motivation
The FFE exposure-generation test intermittently timed out waiting for an exposure. It reused the generic
test-flagkey also used by other FFE coverage and did not verify that the request evaluated the configured variation, making failures ambiguous and susceptible to cross-test state.Changes
onvariation before checking its exposure.Validation
./run.sh FEATURE_FLAGGING_AND_EXPERIMENTATION tests/ffe/test_exposures.py::Test_FFE_Exposure_Events::test_ffe_exposure_event_generation(passed twice with the locally configured weblog)git diff --checkpassed./format.shproceeded through the Python, YAML, and parser checks, then encountered an unrelated macOS Bash 3set -ufailure inutils/scripts/shellcheck.shwhen expanding an empty$@.