Skip to content

Fleet agent update improvements#41

Merged
yusufozturk merged 2 commits into
mainfrom
dev-fleet-update
Jul 2, 2026
Merged

Fleet agent update improvements#41
yusufozturk merged 2 commits into
mainfrom
dev-fleet-update

Conversation

@yusufozturk

@yusufozturk yusufozturk commented Jul 2, 2026

Copy link
Copy Markdown
Member

To test VirtualMetric's Agent update process

Summary by CodeRabbit

  • New Features

    • Added a new fleet automation scenario to validate agent download behavior, including OS/architecture targeting.
    • Introduced configurable per-device download authorization probes with expected HTTP results, plus optional throttling assertions.
  • Bug Fixes

    • Improved scenario validation to require the necessary download-gate and throttling settings before running.
    • Updated the automation flow to skip the prior connection wait for direct download-gate checks and to verify probe histograms consistently, including throttling behavior under concurrency.

@coderabbitai

coderabbitai Bot commented Jul 2, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 381525ce-a47a-4322-86e8-05bbf44fb548

📥 Commits

Reviewing files that changed from the base of the PR and between 6b7db75 and c873610.

📒 Files selected for processing (2)
  • internal/config/case.go
  • internal/runner/runner.go
🚧 Files skipped from review as they are similar to previous changes (2)
  • internal/runner/runner.go
  • internal/config/case.go

Walkthrough

Adds a new agent_download_gate fleet scenario. FleetConfig gains download probe and throttle fields, validation enforces scenario-specific inputs, and the runner adds a simulator probe helper plus scenario logic for per-device status checks and throttling.

Changes

Agent download gate scenario

Layer / File(s) Summary
Config schema and validation
internal/config/case.go
Adds DownloadGate, DownloadOSArch, ThrottleCount, ThrottleMin429, ThrottleDeviceID fields to FleetConfig, introduces DownloadGateProbe, and validates agent_download_gate configuration requirements.
Runner probe and scenario flow
internal/runner/runner.go
Adds fleetSimDLProbe, skips the director-connect wait for this scenario, and implements the agent_download_gate branch that probes /dl, checks expected statuses, and asserts optional throttling.
Estimated code review effort: 3 (Moderate) ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Runner as runFleetAutomationCorrectness
  participant Simulator as Fleet simulator
  participant Director as /dl endpoint

  Runner->>Simulator: fleetSimDLProbe(device 0)
  Simulator->>Director: /dl probe
  Director-->>Simulator: HTTP status
  Simulator-->>Runner: histogram

  Runner->>Runner: wait until histogram["0"] is absent
  loop each DownloadGate probe
    Runner->>Simulator: fleetSimDLProbe(device id)
    Simulator-->>Runner: histogram
    Runner->>Runner: compare observed status with ExpectCode
  end

  opt throttle_count > 0
    Runner->>Simulator: concurrent fleetSimDLProbe calls
    Simulator-->>Runner: histogram with 429 counts
    Runner->>Runner: assert minimum 429 count
  end
Loading

Possibly related PRs

  • VirtualMetric/PipeBench#29: Both extend the same fleet_automation_correctness framework's FleetConfig/validateFleet() and runFleetAutomationCorrectness scenario switch logic.

Suggested reviewers: namles

Poem

A rabbit hops to test the gate,
Counts 200s, 429s, and waits.
With histograms held in paw,
The download path obeys the law. 🐇

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is broadly aligned with the PR, which updates Fleet agent automation and adds download-gate handling.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch dev-fleet-update

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
internal/runner/runner.go (1)

4115-4132: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Histogram-to-status extraction relies on map iteration but is safe under current call pattern.

got is derived by scanning the histogram map for any key with n > 0; with count=1 per probe there should only ever be one such key, so this is correct today, but it's a bit fragile if probe is ever called with count > 1 for per-device checks. Not urgent given current usage.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/runner/runner.go` around lines 4115 - 4132, In the DownloadGate
check inside runner.go, the `got` status is inferred by iterating the histogram
map returned from `probe`, which is only safe because `probe(p.DeviceID, 1, 1)`
currently guarantees a single hit. Make the extraction in this loop
deterministic by choosing the status from the actual probe result structure or
by explicitly validating there is exactly one nonzero bucket before assigning
`got`, so `probe`, `DownloadGate`, and the `/dl` comparison don’t depend on
fragile map iteration if the call pattern changes.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@internal/config/case.go`:
- Around line 1372-1380: The agent_download_gate validation in case.go only
checks download_gate and throttle_count, but it also needs to require the
throttle companion fields when throttling is enabled. Update the validation
logic in the agent_download_gate case to enforce that tc.Fleet.ThrottleMin429 is
greater than zero and tc.Fleet.ThrottleDeviceID is set whenever
tc.Fleet.ThrottleCount is greater than zero, using the existing case.Name and
Fleet fields in this branch so the scenario cannot silently pass with a vacuous
429 check or a blocked default device.

In `@internal/runner/runner.go`:
- Around line 4137-4151: The throttle burst path in runner.go is defaulting to a
blocked device by calling fleetStr(fc.ThrottleDeviceID, "0"), which prevents the
burst from ever reaching the 429 check when throttle_device_id is omitted.
Update the throttle probe setup in the logic around probe and ThrottleCount so
it no longer silently falls back to device "0"; instead, enforce that
ThrottleDeviceID is present whenever ThrottleCount is greater than zero, and
surface a validation error earlier in the flow using the same
fc.ThrottleCount/fc.ThrottleDeviceID fields and the validateFleet-related
handling.
- Around line 4099-4111: The readiness wait around probe in runner.go is
recomputing scenarioDeadline() on every loop iteration, which makes the timeout
slide forward instead of expiring at a fixed settle-based deadline. Snapshot the
deadline once before the loop, then use that fixed value in the waiting
condition so the /dl readiness check fails fast like the other wait sites in
this flow.

---

Nitpick comments:
In `@internal/runner/runner.go`:
- Around line 4115-4132: In the DownloadGate check inside runner.go, the `got`
status is inferred by iterating the histogram map returned from `probe`, which
is only safe because `probe(p.DeviceID, 1, 1)` currently guarantees a single
hit. Make the extraction in this loop deterministic by choosing the status from
the actual probe result structure or by explicitly validating there is exactly
one nonzero bucket before assigning `got`, so `probe`, `DownloadGate`, and the
`/dl` comparison don’t depend on fragile map iteration if the call pattern
changes.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 5c3d7e4c-2ed5-4d9b-9b0b-a8ea4ada6d5f

📥 Commits

Reviewing files that changed from the base of the PR and between 849d015 and 6b7db75.

📒 Files selected for processing (2)
  • internal/config/case.go
  • internal/runner/runner.go

Comment thread internal/config/case.go
Comment thread internal/runner/runner.go
Comment thread internal/runner/runner.go
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 2, 2026

Copy link
Copy Markdown

Deploying pipebench with  Cloudflare Pages  Cloudflare Pages

Latest commit: c873610
Status: ✅  Deploy successful!
Preview URL: https://184eb7c5.pipebench.pages.dev
Branch Preview URL: https://dev-fleet-update.pipebench.pages.dev

View logs

@yusufozturk yusufozturk merged commit f043743 into main Jul 2, 2026
5 checks passed
@yusufozturk yusufozturk deleted the dev-fleet-update branch July 2, 2026 10:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant