Skip to content

Auto-activate ruby easy wins for feature-flagging-and-experimentation-sdk - #7044

Open
dd-octo-sts[bot] wants to merge 1 commit into
mainfrom
easy-win/feature-flagging-and-experimentation-sdk/ruby
Open

Auto-activate ruby easy wins for feature-flagging-and-experimentation-sdk#7044
dd-octo-sts[bot] wants to merge 1 commit into
mainfrom
easy-win/feature-flagging-and-experimentation-sdk/ruby

Conversation

@dd-octo-sts

@dd-octo-sts dd-octo-sts Bot commented May 29, 2026

Copy link
Copy Markdown
Contributor

Automated activation of easy-win tests for ruby owned by feature-flagging-and-experimentation-sdk
View nightly workflow run

  • If you approve this PR please also merge it.
  • If the tests are failing it might be due to a change made since the last nightly system-tests run. You can close the PR, an updated one will be available tomorrow.
  • If you close the PR please also delete the branch

@dd-octo-sts
dd-octo-sts Bot requested review from a team as code owners May 29, 2026 07:11
@dd-octo-sts
dd-octo-sts Bot enabled auto-merge (squash) May 29, 2026 07:11
@github-actions

Copy link
Copy Markdown
Contributor

CODEOWNERS have been resolved as:

manifests/ruby.yml                                                      @DataDog/ruby-guild @DataDog/asm-ruby

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3979338806

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread manifests/ruby.yml Outdated
tests/parametric/test_dynamic_configuration.py::TestDynamicConfigV2::test_tracing_client_tracing_tags: missing_feature # Created by easy win activation script
tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation: v2.23.0
tests/parametric/test_ffe/test_span_enrichment.py: missing_feature
tests/parametric/test_ffe/test_span_enrichment.py: '>=2.34.0' # TODO: a lower version might be supported

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep subject-limit tests disabled until they assert enrichment

With this file-level activation, Test_Span_Enrichment_Max_Subjects is now enabled for Ruby because it is the only subject-enrichment class not listed below as missing_feature. I checked tests/parametric/test_ffe/test_span_enrichment.py: test_max_10_subjects_enforced only validates the limit inside if "ffe_subjects_enc" in meta, so Ruby can emit no ffe_subjects_enc at all and still pass, while the adjacent subject-enrichment classes remain marked unsupported. This makes the easy-win activation report subject enrichment support even when the tracer does not provide it; either keep this class disabled too or make the test fail when the tag is absent.

Useful? React with 👍 / 👎.

@nccatoni
nccatoni disabled auto-merge July 1, 2026 11:53
…tation-sdk

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@dd-octo-sts
dd-octo-sts Bot force-pushed the easy-win/feature-flagging-and-experimentation-sdk/ruby branch from 3979338 to 4b25bc5 Compare August 5, 2026 07:31
@dd-prapprover-prod-77c48c

dd-prapprover-prod-77c48c Bot commented Aug 5, 2026

Copy link
Copy Markdown

PRApprover will approve and merge this PR, FAQ, #dx-source-code-management

🛠️ PRApproval Status

  • ✅ PR is eligible for auto-approval by rule system-tests-auto-activate-easy-wins - 2026-08-05T07:32:20Z
  • ⬜ CI tests passed
  • ⬜ Approved
  • ⬜ Merge Started
  • ⬜ Merged

➡️ Current phase: CI tests failed. Please fix the failing tests and rerun the workflow, click Workflow Link -> Workflow Starter Link -> Start Workflow

@datadog-prod-us1-5

datadog-prod-us1-5 Bot commented Aug 5, 2026

Copy link
Copy Markdown

Pipelines  Tests

⚠️ Warnings

🚦 3 Pipeline jobs failed

Testing the test | System Tests (ruby, dev) / End-to-end #1 / rails72 1   View in Datadog   GitHub Actions

🔧 Fix in code. This looks caused by changes in this PR. 2 failed tests due to assertion errors: timeout waiting for exposure event and EVP flagevaluation count.

🧪 2 Tests failed

tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Degradation.test_ffe_evp_flagevaluation_degradation[rails72] from system_tests_suite   View in Datadog
AssertionError: Timed out waiting for EVP flagevaluation count &gt;= 12000 for flag evp-degradation-flag
assert False
 &#43;  where False = &lt;bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator(&#39;agent&#39;)&gt;(&lt;function wait_for_evp_flagevaluation_count.&lt;locals&gt;.&lt;lambda&gt; at 0x7f3c5b116980&gt;, timeout=60)
 &#43;    where &lt;bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator(&#39;agent&#39;)&gt; = AgentInterfaceValidator(&#39;agent&#39;).wait_for
 &#43;      where AgentInterfaceValidator(&#39;agent&#39;) = interfaces.agent

self = &lt;tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Degradation object at 0x7f3c89cae4b0&gt;

    def test_ffe_evp_flagevaluation_degradation(self) -&gt; None:
        for index, response in enumerate(self.responses):
...
❄️ Known flaky: tests.ffe.test_exposures.Test_FFE_EXP_5_Missing_Targeting_Key.test_ffe_exp_5_missing_targeting_key[rails72] from system_tests_suite   View in Datadog
AssertionError: Timed out waiting for exposure event for flags [&#39;exp-5-missing-targeting-key-flag&#39;] and subject &#39;&#39;
assert False
 &#43;  where False = &lt;bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator(&#39;agent&#39;)&gt;(&lt;function wait_for_exposure_event.&lt;locals&gt;.&lt;lambda&gt; at 0x7f3c61a68cc0&gt;, timeout=30)
 &#43;    where &lt;bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator(&#39;agent&#39;)&gt; = AgentInterfaceValidator(&#39;agent&#39;).wait_for
 &#43;      where AgentInterfaceValidator(&#39;agent&#39;) = interfaces.agent

self = &lt;tests.ffe.test_exposures.Test_FFE_EXP_5_Missing_Targeting_Key object at 0x7f3c89c630b0&gt;

    def test_ffe_exp_5_missing_targeting_key(self):
        &#34;&#34;&#34;EXP.5: Test that empty targeting key generates exposure with subject.id = &#39;&#39;.&#34;&#34;&#34;
...

Not introduced in this PR.

Testing the test | System Tests (ruby, prod) / End-to-end #1 / rails72 1   View in Datadog   GitHub Actions

🔧 Fix in code. This looks caused by changes in this PR. 2 failed tests due to assertion errors in flag evaluation: timed out waiting for exposure event and EVP flagevaluation count.

Testing the test | all-jobs-are-green   View in Datadog   GitHub Actions

See error 2 check runs failed for System Tests in ruby environments.
📋 Copy prompt for your agent
CI on my pull request is failing. Help me find and fix the root cause of each failing job below — they were flagged as caused by changes in this PR, so focus on the diff. For each job, explain the failure and propose a fix.

Branch: easy-win/feature-flagging-and-experimentation-sdk/ruby

Testing the test | System Tests (ruby, dev) / End-to-end #1 / rails72 1
Commit: 6c54a822059a1cfc72be878df5de0b54f9c9b6bc
Error (code / test):
2 failed tests due to assertion errors: timeout waiting for exposure event and EVP flagevaluation count.
CI job: https://github.com/DataDog/system-tests/actions/runs/30985394879/job/92240185765

Testing the test | System Tests (ruby, prod) / End-to-end #1 / rails72 1
Commit: 6c54a822059a1cfc72be878df5de0b54f9c9b6bc
Error (code / test):
2 failed tests due to assertion errors in flag evaluation: timed out waiting for exposure event and EVP flagevaluation count.
CI job: https://github.com/DataDog/system-tests/actions/runs/30985394879/job/92240138755

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 4b25bc5 | Docs | Datadog PR Page | Give us feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants