Skip to content

Render hunt-page ruleset tracking and hunt provenance fields - #266

Merged
sbneto merged 54 commits into
developfrom
DN-8480-hunting-schema-migration
Aug 31, 2026
Merged

sbneto merged 54 commits into
developfrom
DN-8480-hunting-schema-migration

Conversation

@vhmartinezm

@vhmartinezm vhmartinezm commented Aug 21, 2026 •

Copy link
Copy Markdown
Contributor

TL;DR

The CLI leg of the hunt-page capability: render the ruleset tracking and hunt provenance fields the SDK now parses, and add rules favorite <id> [--unfavorite] — the star toggle the platform delivery-order standard requires this change set to carry. Formatter legs read the fields directly; the pin guarantees the SDK parses them, so None means the server had no answer.

Requires

  • Hunt-page ruleset tracking: favorites, rule counts, hunt provenance polyswarm-api#321 — merge that to develop first, and release it first. This PR pins polyswarm_api>=4.4.0, the version that PR declares. Two distinct orderings follow:
    • Merge in lockstep. The two develop branches are tested against each other by design, so a change spanning the pair lands on both together. Merging this side alone leaves its develop asking for an SDK develop.zip that does not yet declare the floor, and the install fails — the pairing being broken, not a trap to design around.
    • Publication is a later cutoff. Merging to develop publishes nothing; PyPI only sees a version at develop → master. At that point the SDK version is confirmed (bumped if the repo files do not already carry it) and this repo's dependency is set to the version actually released. The >=4.4.0 here is the working value develop integration validates.
      This branch's own CI is unaffected either way — it resolves the paired SDK branch by name.

What's new

  • rules favorite <id> [--unfavorite]: renders the toggle state, Favorited at, and the server-owned budget ("Favorites used: N of M" — the client never counts). A full-budget refusal (the server's machine-readable FAVORITE_LIMIT) becomes a clean actionable message at exit 2 — the central mapping's server-refusal code; exit 1 stays reserved for no-results/not-found.
  • The floor moves to 4.4.0 — the version introducing ruleset_favorite. No getattr, no upgrade message: the pin makes an SDK without it uninstallable, so the command calls it directly.
  • ruleset: Favorite: yes (+ Favorited at), Rules in ruleset (omitted when the server had no answer — never shown as 0), Historical hunts triggered, and New live results — the server's stored badge, deliberately unlabelled with a window since the response carries none and a caller cannot choose one — with the new_results_counted_at staleness marker beside it, which is what tells a reader how current the number is. (The earlier --include-counts flag is withdrawn with the per-request server aggregate it wrapped, per the platform query-design standard.)
  • rules list gains the four server-side filters --name / --status active / --favorites-only / --has-new-results. The list is keyset-paginated, so filtering locally would mean walking every page. An unfiltered rules list is still a zero-argument ruleset_list() call — a False flag is not a filter.
  • live feed gains --livescan-id — the drill-down for the per-ruleset new-results badge, which had no way to list the results it counts — and --max-results (unset or 0 means no bound, as before). --since keeps its seconds unit and moves its default from 1440 to 86400 — the 24h it was always written for (the old value was 24 * 60, against an SDK docstring that wrongly said minutes). The wire is untouched: this endpoint takes ~197k requests per 30 days carrying since from clients outside our control, so re-basing it to minutes would widen every one 60x silently. The CLI's own default window does widen 60x (24 min -> 24 h) as a result — worth a release note on the develop -> master PR, since a plain live feed now returns a day of results; 0 still means no time filter at all.
  • No runtime SDK probing — the pin is the contract. The CLI calls the new surfaces directly: no signature inspection, no getattr fallbacks, no per-test skip guards. A probe would restate what the pin already says with nothing keeping the two in sync, and it fails in both directions — check less than the call site uses and execution reaches the missing surface; check more and the fallback fires while the surface is present, which in a test is a silent skip that drops coverage. specs/04 and specs/05 carry the rule; the workspace repo carries it as an org-wide standard.
  • Coverage goes up: 146 tests, none skipped. Every test that could previously skip itself now always runs.
  • hunt: Source Ruleset Id, the source's last-modified at freeze time, and Source ruleset changed since this hunt froze it: yes/no — the label names the reference point deliberately; unknown prints nothing.

Tests

tests/formatter_hunt_fields_test.py pins the render legs against real SDK resources, zero-distinct-from-absent for the counters, the truthy-only favorite leg, the staleness-marker render, the reference point in the changed-since-freeze label, the zero-argument ruleset_list() call (signature-checked via autospec), the FAVORITE_LIMIT exit-2 message, and the exception hierarchy the exit-code mapping depends on. The ruleset-shaped VCR cassettes are re-recorded against a stack running the paired server branch (hunt provenance stays null in every recording, so those render legs are unit-pinned rather than cassette-pinned) (every ruleset body carries the tracking keys; list bodies carry the stored counter pair; livescan_id renders as a string when present), with new recordings for the favorite/unfavorite round-trip.

Two formatter legs, both getattr-guarded so the CLI still renders results
parsed by an SDK release that predates the fields:

- ruleset: Favorite / Favorited at, Rules in ruleset (absent when the
  server had no answer — never shown as 0), Historical hunts triggered,
  and New live results in window (only when the caller asked the list to
  include counts).
- hunt: Source Ruleset Id, the source's last-modified at freeze time, and
  'Source ruleset changed since this hunt froze it: yes/no' — the label
  names the reference point deliberately; unknown prints nothing.
The getattr guards (an old-SDK result without the attributes renders,
new lines omitted), zero-distinct-from-absent for the counters, the
truthy-only favorite leg, and the reference point in the
changed-since-freeze label.
Maps to the server's include_counts so the 'New live results in window'
formatter leg is reachable from the CLI (only live-hunting rulesets
carry a count; the param is omitted unless asked).
The flag must reach ruleset_list(include_counts=True) and the unflagged
run must omit the param entirely — the SDK drops None, and the exact
wire value is load-bearing (the server only accepts '0'/'1'/'false'/
'true').
@claude

claude Bot commented Aug 21, 2026

Copy link
Copy Markdown

Review. Base is develop OK, commit messages carry no ticket IDs OK. Four things need action, one of them blocking.

1. Blocking — --include-counts is a hard SDK dependency the pin cannot express (src/polyswarm/client/rules.py:41)

The formatter legs are getattr-guarded, but this call site is not:

for ruleset in api.ruleset_list(include_counts=include_counts or None):

ruleset_list() takes no arguments before polyswarm-api PR 321, and that PR does not bump the SDK version — the SDK pyproject still declares 4.3.0. This repo floors at polyswarm_api>=4.3.0,<5.0.0, which the published 4.3.0 satisfies while lacking the kwarg. So once this merges, pip install polyswarm-cli against SDK 4.3.0 turns a plain polyswarm rules list — no flag — into TypeError: ruleset_list() got an unexpected keyword argument, which ExceptionHandlingGroup renders as a traceback plus "Unhandled exception happened. Please contact support." and exit 2. Arguments bind at call time even for a generator function, so nothing defers it to iteration.

CI will be green (the SDK branch name matches this branch, so the archive install picks up 321), which is why this needs catching in review rather than from a red pipeline.

specs/05-sdk-contract.md: "The SDK version pin in pyproject.toml is the compatibility contract. Floor it at the lowest SDK version that exposes every method/behaviour the CLI relies on" — and a floor bump has two preconditions there: the version is on PyPI, and the SDK develop declares at least that version. Neither holds today. The PR body reasoning "No version-pin bump: the getattr guards are exactly the degradation path" is true of the formatter fields and not of this line.

Two ways out:

  • have PR 321 bump the SDK to 4.4.0, release it, then floor this PR at >=4.4.0; or
  • keep the default path degradable, matching the claim the PR body already makes — build the kwargs conditionally (dict(include_counts=True) when flagged, empty otherwise) and splat them, so rules list keeps working on 4.3.0 and only the new flag needs the new SDK.

(The or None itself is fine — the existing test_ruleset_list_json cassette pins that the SDK drops None params, since the default query matcher would reject an added include_counts key.)

2. The new mock hides exactly that failure (tests/formatter_hunt_fields_test.py:96)

mock.patch(...PolyswarmAPI.ruleset_list) without autospec=True replaces the method with a signature-free MagicMock, so test_flag_sends_include_counts_true and test_no_flag_omits_the_param both pass green against an SDK whose ruleset_list() accepts no arguments. Add autospec=True and the test becomes a real signature check against the installed SDK — the one thing that would have surfaced (1) locally.

3. Formatter tests use SimpleNamespace, so field renames fail silently (tests/formatter_hunt_fields_test.py:29-40)

Every new line is getattr-guarded, which converts an attribute-name mismatch into silent omission rather than an error. Fed hand-built namespaces, these tests stay green if the SDK renames historical_hunt_count or source_rule_changed and the CLI quietly stops rendering it. specs/04-testing.md Style 3 asks for "an SDK resource built from a literal dict" — known_good_field_test.py does that with ArtifactInstance. Constructing real resources.YaraRuleset / resources.HistoricalHunt objects from literal dicts here couples the guards to the real attribute names, and additionally pins that favorited_at / rule_modified arrive as parse_isoformat datetimes rather than the raw strings the tests currently feed.

4. Spec drift — no spec touched

AGENTS.md, step 6: "Update the specs for the area you touched (at least 02-commands.md) in the same PR." rules list grew a user-facing option and the rules row in specs/02-commands.md does not mention it. specs/03-formatters.md says rendering rules get documented "when they become non-obvious or contested" — the semantics this PR encodes in code comments are exactly that: rule_count=None is not 0, favorite is truthy-only (so "not favorited" and "old SDK" render identically), source_rule_changed is tri-state with None meaning unknown rather than unchanged, and the label deliberately names the freeze as its reference point.

5. Minor (src/polyswarm/formatters/text.py:301)

if getattr(result, 'favorite', None) is not None and result.favorite: — the is not None half is dead, since None is falsy. if getattr(result, 'favorite', None): says the same thing.

…sources

Review findings:

- Blocking: the unconditional include_counts= kwarg made plain
  'rules list' a hard dependency on an SDK newer than the pin's floor —
  4.3.0's ruleset_list takes no arguments, so every unflagged run would
  TypeError against the published SDK (CI could not see it: the branch
  archive install picks up the new SDK). The kwargs are now built
  conditionally; only --include-counts requires the new SDK, matching
  the degradation claim the PR body makes.
- The flag tests now autospec the mock, turning both assertions into
  signature checks against the installed SDK — the check that would
  have caught the above locally.
- The rendering tests build REAL SDK resources from literal dicts, so
  an SDK attribute rename fails the test instead of silently dropping a
  line (the getattr guards convert mismatches into omission); they also
  pin that favorited_at/rule_modified arrive as parsed datetimes.
  SimpleNamespace remains only for the old-SDK degradation cases, where
  absent attributes are the point.
- specs/02-commands.md documents the new flag and the floor-SDK
  constraint; specs/03-formatters.md records the non-obvious rendering
  semantics (0-vs-None, truthy-only favorite, the tri-state and its
  reference point). Dead 'is not None' half of the favorite guard
  dropped.
@claude

claude Bot commented Aug 21, 2026

Copy link
Copy Markdown

Review — hunt-page tracking fields

Two things need action; the formatter legs themselves look right.

1. --include-counts needs a pin bump, and getattr guards don't cover it (correctness / spec drift)

specs/05-sdk-contract.md §Invariants: "The SDK version pin in pyproject.toml is the compatibility contract. Floor it at the lowest SDK version that exposes every method/behaviour the CLI relies on."

The PR body says "No version-pin bump: the getattr guards are exactly the degradation path for the current polyswarm_api>=4.3.0,<5.0.0 range." That reasoning holds for the two formatter legs — a missing attribute silently omits a line. It does not hold for src/polyswarm/client/rules.py:45. The comment right above it states the problem itself:

an installed SDK at the pin's floor (4.3.0) has a zero-argument ruleset_list

So on a conforming install (polyswarm_api==4.3.0, which the pin explicitly permits), polyswarm rules list --include-counts hits TypeError: ruleset_list() got an unexpected keyword argument 'include_counts'. ExceptionHandlingGroup has no branch for that — it falls through to the terminal except Exception (client/polyswarm.py:164) and the user gets a full logger.exception traceback plus "Unhandled exception happened. Please contact support." at exit 2. The conditional-kwarg trick protects plain rules list, not the flag the PR is adding; a flag that is guaranteed to crash on the declared floor is a flag the pin doesn't cover.

Per spec 05 the floor has two preconditions before it can move (the version is on PyPI, and the SDK's develop declares at least that version, no .devN suffix). If polyswarm-api#321's release satisfies both, bump the floor here. If it doesn't yet, the flag can't ship in this PR — either way the PR needs an explicit bump decision rather than "no bump", and the "keeps working on the pin's floor SDK" claim now in specs/02-commands.md:33 needs to say the same thing.

2. specs/05-sdk-contract.md §"Current floor" is stale (spec drift)

The section header and body still read polyswarm_api>=4.2.0, but pyproject.toml:25 has said >=4.3.0 since #264. Pre-existing, but this PR's entire no-bump argument (and the new comment in rules.py) is reasoning off "the pin's floor", and the PR already edits two specs — fix it here rather than leave the authoritative doc contradicting the file it documents. Whatever lands for #1 goes in the same section.

Minor

  • specs/04-testing.md §Style 3 asks for TextOutput(color=False) called with write=False, asserting on the returned lines — no stream. tests/formatter_hunt_fields_test.py renders through an io.StringIO instead. Equivalent in effect, but it diverges from the convention known_good_field_test.py sets; worth matching for consistency.

Clean

  • Base is develop, no CLI version bump, no ticket IDs in commit messages or PR text — gitflow and hygiene rules all satisfied.
  • JSONOutput correctly needs no change (both hunt and ruleset dump result.json).
  • Zero-vs-absent handling, the truthy-only favorite leg, and the tri-state source_rule_changed guard all match the invariants the PR adds to specs/03-formatters.md, and each is pinned by a test.
  • Building the render fixtures from real resources.YaraRuleset / resources.HistoricalHunt instances is the right call — with getattr guards, a hand-built namespace would turn an attribute-name mismatch into a silently passing test.

@vhmartinezm

Copy link
Copy Markdown
Contributor Author

All five addressed in ff1dc77: the blocking one is fixed as suggested — the kwargs are built conditionally, so plain rules list keeps working on the pin's floor SDK and only --include-counts needs the new one (specs/02 documents that constraint); the flag tests are autospec'd (the signature check that would have caught it locally); the rendering tests now build real resources.YaraRuleset/HistoricalHunt from literal dicts — SimpleNamespace remains only for the old-SDK degradation cases, where absent attributes are the point — and pin the parse_isoformat datetimes; specs/03-formatters records the 0-vs-None, truthy-only-favorite and tri-state semantics; the dead is not None half is gone.

@vhmartinezm
vhmartinezm requested a review from sbneto August 24, 2026 16:53
@sbneto

sbneto commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Review — hunt-page rendering, measured against the platform query-design and delivery-order standards

Reviewed against our org-wide project standards — §13 Query design (no aggregate computed on a request path; client-visible counts are stored columns refreshed by a scheduled job with an observable staleness marker) and §14 Delivery order (a capability ships API → SDKs and CLI → UI; API, SDK and CLI support land in one change set) — plus the design decisions settled on the server side of this change.

What's clean. The rendering legs are careful and the semantics are documented where they are non-obvious rather than left in code comments: 0 distinct from None on both counts, truthy-only favorite, and the tri-state source_rule_changed whose label names its reference point — "changed since this hunt froze it" — so it cannot be misread as "edited recently". Building the render fixtures from real SDK resources rather than namespaces is the right call, since the getattr guards would otherwise turn an attribute rename into a silently passing test, and keeping SimpleNamespace only for the old-SDK degradation cases is the correct exception. JSONOutput needing no change is right. Base develop, no version bump, no internal ticket id in the title, body or commits. Nothing here aggregates client-side.


Findings

[MODERATE] F1. --include-counts is built on a server aggregate that is being withdrawn — and crashes on the declared floor SDK today

What happens: Two problems in one line. The flag calls ruleset_list(include_counts=True), which is a TypeError on the published SDK 4.3.0 that this repo's pin explicitly permits — ExceptionHandlingGroup renders that as a traceback plus "Please contact support" at exit 2. And the parameter it passes wraps a per-request COUNT … GROUP BY on the server, which is a design flaw under §13 and is being replaced by a stored counter column refreshed by a scheduled job. So the flag is both unsafe on a conforming install and pointed at a parameter that is going away.

When: The crash fires on any rules list --include-counts against an install at the pin's floor — which CI cannot surface, because the branch-name SDK archive install always picks up the matching branch.

Why:

  • The conditional-kwargs fix protects plain rules list but not the flag itself; a flag guaranteed to fail on the declared floor is a flag the pin does not cover.
  • specs/05-sdk-contract.md makes the pin the compatibility contract, and moving the floor has two preconditions — the version on PyPI, and the SDK's develop declaring at least that version — neither of which holds.
  • §13's first rule puts client-visible counts in stored columns refreshed asynchronously; the window becomes a property of the refresh job rather than of the request.

Proposed fix (untested): remove --include-counts, the conditional-kwargs machinery (list_rules returns to api.ruleset_list()), both tests in RulesListIncludeCountsFlagTest, and the flag's description from the rules row in specs/02-commands.md. That resolves the floor problem outright rather than deferring it: with ruleset_list() back to zero-arg, this PR needs no new SDK behaviour at all — every field it renders is getattr-guarded — so no pin bump is required and the whole floor question goes away.

new_results_count itself stays and keeps rendering; it becomes a stored column that is always present, None meaning "not yet refreshed".

[MODERATE] F2. The favorite capability ships with no CLI leg

What happens: A user can favorite a ruleset through the API and through the SDK, but not through the CLI. The companion SDK PR adds ruleset_favorite(); this PR touches only list_rules, and the rules row in specs/02-commands.md still lists ruleset_{create,delete,update,get,list}.

When: On merge — the capability is simply absent from this interface.

Why:

  • §14 requires API, SDK and CLI support for a capability to land in one change set. This one has two of three legs.
  • The endpoint returns favorites_used / favorites_limit specifically so a client can render "N of M used" without counting — a contract with no consumer here.
  • The refusal path is machine-readable through exc.request.errors['code'] for the same reason, and nothing in this repo reads it.

Proposed fix (untested): add rules favorite <id> with an --unfavorite flag, rendering the returned star state plus the two counters, and handling the FAVORITE_LIMIT refusal as a clean message rather than a traceback. Add its row to specs/02-commands.md and its rendering rules to specs/03-formatters.md beside the ones this PR already documents.

  • [LOW] F3. "New live results in window" names a window the caller can no longer choose or see, once the window belongs to the refresh job. Fix (untested): name the fixed product window in the label, and render the new_results_counted_at staleness marker the server will expose beside it — §13 requires that marker to be observable, which means the interface has to show it.
  • [LOW] F4. specs/05-sdk-contract.md:77 still headers polyswarm_api>=4.2.0 while pyproject.toml:25 has said >=4.3.0 since release: bump version to 4.3.0, floor the SDK at 4.3.0 #264. Pre-existing, raised in the 21 Aug review and not addressed — and it matters here specifically, because this PR's whole no-bump argument reasons off "the pin's floor" while the authoritative doc names a different one. Fix (untested): correct the header and body in this PR, since it already edits two specs.
  • [LOW] F5. The ruleset cassettes still mirror the contract this change alters: three carry livescan_id as a bare int (test_live_hunt_start_json.click, test_live_hunt_start_json.vcr, test_live_hunt_start_text.vcr), and all 16 ruleset-shaped recordings predate the four tracking keys, while the JSON formatter dumps the body verbatim. The companion SDK PR re-recorded its two cassettes; this one touches none, so replay stays green while a live run diverges. Fix (untested): re-record on this branch. Needs confirmation: that requires a live stack running the server branch's image, which was not stood up here.

Standards conformity

§14 — conformant on ordering, incomplete on coverage. This is correctly the third leg, on the identical branch name, based on develop, merging after the SDK. F2 is the gap: one of the two capabilities in this change set has no CLI support.

§13 — F1 is the gap, inherited from the server design this PR wrapped rather than introduced here. Removing the flag brings this repo into line and removes the floor hazard at the same time.

Fixes are proposed, not applied; nothing was run.

@vhmartinezm

Copy link
Copy Markdown
Contributor Author

Review applied — head e391f46

  • F1: --include-counts removed outright — flag, conditional-kwargs machinery, both tests, spec row. list_rules is zero-argument again (signature-checked via autospec) and the badge renders from the stored counter fields.
  • F2: rules favorite <id> [--unfavorite] added: renders state + the server-owned "Favorites used: N of M", and the machine-readable FAVORITE_LIMIT refusal becomes a clean actionable message at exit 2 (the central mapping's server-refusal code — a ClickException's default 1 would collide with no-results). One consequence of adding the leg in this change set: ruleset_favorite doesn't exist on published 4.3.0, so the command getattr-guards and degrades to a clean upgrade message on a floor install — every pre-existing command still works on the floor, no pin bump (specs/05 documents the exception and the follow-up bump once the SDK releases). VCR recordings cover the favorite/unfavorite round-trip.
  • F3: the label names the fixed window ("New live results (last 24h)") and the new_results_counted_at staleness marker renders beside it.
  • F4: the specs/05 floor header now follows the pin (>=4.3.0), with a note on why the floor lives in one authoritative place.
  • F5: all hunt-shaped cassettes (ruleset CRUD, live start/stop, historical create/delete/list) re-recorded against a stack running the server branch's image — every ruleset body carries the tracking keys, list bodies carry the stored counter pair, livescan_id is a digit string throughout.

@claude

claude Bot commented Aug 25, 2026

Copy link
Copy Markdown

Reviewed against AGENTS.md and specs/01–05. The change matches the documented conventions: formatter method added to base.py/text.py/json.py (§03 invariant), getattr-guarded legs, rules list back to a zero-argument ruleset_list(), server-refusal → exit 2 per the §01 mapping, ## Requires link present, base is develop, no version bump. Three items:

1. Merge gate (already known, restating because it is load-bearing). polyswarm/polyswarm-api#321 is still open. Per specs/05-sdk-contract.md §Coordinated changes this must not merge until ruleset_favorite / YaraRulesetFavorite are on the SDK's develop. The PR build is fine — the SDK branch name matches, so $CI_COMMIT_BRANCH.zip resolves — but a merge to develop falls back to the SDK's develop.zip, and every favorite test errors at patch time.

2. tests/formatter_hunt_fields_test.py cannot run on the floor SDK it certifies. mock.patch("polyswarm_api.api.PolyswarmAPI.ruleset_favorite", …) (4 tests) and resources.YaraRulesetFavorite(...) (3 tests) both raise AttributeError against published polyswarm_api 4.3.0 — the exact install the rules favorite guard exists for, and one the pin permits. test_favorite_on_the_floor_sdk_degrades_cleanly is the sharpest case: it claims to simulate the floor but can only run where the attribute already exists. create=True on that patch, plus skipUnless(hasattr(PolyswarmAPI, "ruleset_favorite"), …) on the other four, makes the suite honest on both installs.

3. Ticket id in the branch name (DN-8480-…). CLAUDE.md bans ticket ids from commit messages, PR titles and descriptions on this public repo; a merge commit embeds the branch name in history — squash-merge with a clean subject, or rename the branch. Same spirit: commit dab6c5e cites internal standard section numbers ("org delivery-order standard §14", "query-design standard §13") in public history.

Nothing else. The cassettes are genuine re-recordings (the 3.0.0-era 308-redirect interactions are gone, UA is 4.3.0, ids are this run's resources), the .click fixtures line up with the recorded bodies — new_results_count is null in the list bodies and absent from the detail serializer, so the badge and provenance legs render only in the Style-3 unit tests, which is the sanctioned split — and rule_id/source_rule_changed are null throughout because historical start <file> is a raw-yara hunt. The FAVORITE_LIMIT handling matches the SDK's documented envelope (exc.request.errors), including the deliberate absence of a typed exception.

@vhmartinezm
vhmartinezm force-pushed the DN-8480-hunting-schema-migration branch from e391f46 to 04dd0ac Compare August 25, 2026 22:37
@vhmartinezm

Copy link
Copy Markdown
Contributor Author

Applied at head:

  1. Merge gate acknowledged — the SDK PR merges first, per its ## Requires and the coordinated-changes rule; nothing merges until ruleset_favorite is on the SDK's develop.
  2. The suite is now honest on both installs: every test touching the new surface (ruleset_favorite patches, YaraRulesetFavorite fixtures) carries skipUnless(hasattr(...)), and the floor-degradation test patches with create=True so it runs — and means something — on published 4.3.0 too.
  3. Commit messages reworded without the internal section numbers; squash-merge with a clean subject on the day covers the branch name.

@claude

claude Bot commented Aug 25, 2026

Copy link
Copy Markdown

Review

Gitflow is clean (base develop, no pyproject.toml version bump, ## Requires links the SDK PR), the formatter methods land on BaseOutput/TextOutput/JSONOutput as specs/03 requires, and the floor exception is documented in specs/05. Findings below, most severe first.

1. tests/cli_test.py favorite cassettes break the "honest on both installs" claim.
test_ruleset_favorite_text / test_ruleset_unfavorite_text have no _needs_favorite_sdk guard. On an install that satisfies the declared pin (published polyswarm_api==4.3.0, which specs/05 now explicitly documents as supported), rules favorite takes the toggle is None branch and exits 2 with the upgrade message — both cassette tests then fail against their .click. formatter_hunt_fields_test.py goes out of its way to skip there; these two do not. Either apply the same skipUnless or drop the claim.

Related: _SDK_HAS_FAVORITE also requires hasattr(resources, "YaraRulesetFavorite"). If the paired SDK names the resource anything else, all five favorite tests silently skip and CI stays green with the new command effectively untested (only the floor-degradation test still runs). The command tests only need ruleset_favorite to exist — keying the skip on the method alone shrinks that blind spot.

2. The FAVORITE_LIMIT wire shape is load-bearing and nothing pins it. src/polyswarm/client/rules.py:79-81 reads exc.request.errors["code"]. Two problems:

  • specs/05 §"What the CLI imports from the SDK" enumerates the SDK exceptions the CLI consumes (NoResultsException, NotFoundException, FailedInstanceException, PolyswarmException). RequestException, and the structured .request.errors payload behind it, are a new coupling to the SDK error surface that this PR spec edits do not record. Add it to that table (and to §Current floor if it needs the paired SDK).
  • test_favorite_limit_refusal_is_a_clean_message_at_exit_2 builds request as a bare mock.Mock() and assigns .errors itself, so the test passes regardless of where the SDK actually puts the server error body or what the code string is. A rename on either side silently disables the friendly message (the bare raise falls through to a raw SDK message at exit 2) with no test failure. specs/03 §Known-good sets the precedent for exactly this situation — state where the key was read off the server serializer, and where it is pinned. A recorded 4xx-refusal cassette would pin it properly.

3. No coverage for rules favorite --output-format json. specs/04 §"What to test for a new command" item 2 asks for both formats where both matter. JSONOutput.ruleset_favorite assumes the new resource exposes .json; nothing in the suite exercises that path (both new cassettes and all unit tests are text-only).

4. Question on the re-recorded rules view body. tests/vcr/test_ruleset_view_json.click loses "community": "_public" — the detail response no longer carries it. That is a user-visible change to rules view --output-format json output and it is not mentioned in the PR body. Confirm it is an intentional server-side serializer change on the paired branch and not an artefact of the recording stack.

5. Minor, gitflow hygiene. AGENTS.md bans internal ticket IDs in commit messages / PR titles / descriptions; the branch name would land its ticket code in public history via a default merge-commit subject. Squash-merge with a clean subject.

…ntract

- rules favorite <id> [--unfavorite]: the CLI leg of the favorite
  capability (API, SDK and CLI land together as one change set).
  Renders the toggle state plus the server-owned 'Favorites used: N of
  M' budget, and converts the machine-readable FAVORITE_LIMIT refusal
  into a clean actionable message at exit 2 — the central mapping's
  server-refusal code (a ClickException would exit 1, the code reserved
  for no-results/not-found). Pinned end-to-end against a real recorded
  400 (tests/cli_test.py::test_ruleset_favorite_limit_text), not just a
  hand-built mock, so a rename of the error shape on either side fails
  a test.
- The command guards the SDK surface: ruleset_favorite ships in the
  paired SDK change and does not exist on the declared floor
  (published 4.3.0), so on a floor install the command fails with a
  clean upgrade message instead of an AttributeError traceback — the
  same only-the-new-surface-may-require-the-new-SDK principle as the
  withdrawn flag below. Every pre-existing command works unchanged on
  the floor; the floor itself cannot move until the SDK releases
  (specs/05 documents the exception and the follow-up bump). Every
  test touching the new surface is guarded on the narrowest dependency
  it actually needs (the method for command tests, the resource class
  for formatter fixture tests) so a rename on either side skips only
  the tests that need it, not the whole suite silently.
- drop --include-counts: it wrapped a per-request server aggregate that
  is withdrawn (no count is computed on a request path; the badge is a
  stored, server-refreshed counter) — and it crashed on the declared
  floor SDK, which CI's branch-name SDK install could never surface.
  With it gone, list_rules is zero-argument again.
- the new-results badge renders as 'New live results (last 24h)' — the
  fixed product window, since a caller can no longer choose one — with
  the new_results_counted_at staleness marker beside it.
- specs: the sdk-contract floor header follows the pin (>=4.3.0, moved
  by #264; the header had lagged at 4.2.0), the imports table records
  RequestException/.request.errors as a real SDK dependency, and the
  command/formatter tables cover the favorite leg, its floor
  degradation, and the stored-counter render.
All ruleset/live/historical cassettes (and their click snapshots)
re-recorded against a stack running the paired server branch: every
ruleset body carries the four tracking keys, the LIST bodies
additionally carry the stored new_results_count / new_results_counted_at
pair (the detail serializer deliberately does not render the badge),
livescan_id is a digit string on every surface, and the fixture ids are
this recording run's own resources. New recordings cover the favorite
toggle's happy path in both output formats, unfavorite, and a real
FAVORITE_LIMIT refusal recorded against a genuinely full team budget.
@vhmartinezm
vhmartinezm force-pushed the DN-8480-hunting-schema-migration branch from 04dd0ac to b430a14 Compare August 26, 2026 15:03
@vhmartinezm

Copy link
Copy Markdown
Contributor Author

Applied at head:

  1. Both existing favorite cassette tests now carry _needs_favorite_method, and two new recordings were added: test_ruleset_favorite_json (the JSON leg you flagged missing) and test_ruleset_favorite_limit_text — a real FAVORITE_LIMIT 400 recorded against a genuinely full team budget (5 rulesets starred, then the 6th refused), all four guarded the same way. Also split the guard into _needs_favorite_method / _needs_favorite_resource per your note — command tests key on the method alone so a resource rename can't silently skip the whole command suite.
  2. The FAVORITE_LIMIT wire shape is now pinned by that real cassette rather than only the hand-built mock (the mock tests stay, for the exit-code/message-formatting unit coverage) — a rename on either side fails test_ruleset_favorite_limit_text. Also added the RequestException/.request.errors line to specs/05's imports table.
  3. Covered above (recording a few fixes, since parameter #1).
  4. Checked — not a recording artifact. community was never emitted by the server's YaraRuleSerializer (the ruleset detail view), on this branch or on master; rules view's JSON output doesn't carry it under either. Confirmed by diffing the serializer against origin/master (zero community references in either version of that class) — nothing to fix here.
  5. Agreed, squash-merge with a clean subject on the day.

@sbneto

sbneto commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds the hunt page's backend: a team-shared favorite star with a 5-slot budget, ruleset-list filters, a per-hunt "new results" badge kept in a stored column by a scheduled job, a per-hunt feed scope, and provenance linking each historical hunt back to the ruleset it froze — plus the SDK and this CLI that read all of it, and the deploy entries that run the jobs. 4 PRs on the shared branch name; 93 files, +7320/−1277 across the set. This member is 47 files, +1381/−873.

Severity: 0 HIGH · 4 MODERATE · 6 LOW. Prior feedback: 54 checked · 4 open.
Objective: met, with gaps — the hunt page's tracking columns, their write/read surface, and the SDK/CLI legs.

Fixes are proposed, not applied; nothing was run; findings verified by single-pass reading against the current heads.

Cross-repo coordination

Merge order: internal API → polyswarm/polyswarm-api#321 → this PR, with the chart deployed alongside the server image. §14: the API is the contract.

Surface Producer Consumer
?livescan_id= feed scope internal API polyswarm-cli → F4
live_feed(since=) unit internal API polyswarm-cli → F5

Coherence: F4 and F5 both add options to live feed and share one paired-SDK guard — apply them together, not in sequence. F5's server half must land first, or the CLI will express minutes while the wire still reads seconds.

This PR carries the two most important findings in the set.

Findings (round 2)

Every finding below is work for this change set; each entry's Lands in: names the repo whose PR carries the fix, on the branch name every member already shares.

[MODERATE] F4. The CLI renders the hunt badge but cannot list the results it counts

What happens: polyswarm rules view now prints New live results (last 24h): N for a ruleset, and there is no way to see those N results from the CLI. live feed has no option to scope to one hunt, so the badge's own drill-down is unreachable. The four new ruleset-list filters have no CLI surface either.

When: On merge — the capability is simply absent from this interface.

Why:

  • The server added a livescan_id filter to the live-results list, and the SDK exposes it as live_feed(livescan_id=) (src/polyswarm_api/api.py:597 in the paired PR).
  • The CLI's feed command forwards five kwargs and not that one (src/polyswarm/client/live.py:45).
  • rules list calls a zero-argument ruleset_list() (src/polyswarm/client/rules.py:45) while the SDK gained name, status, favorites_only and has_new_results.
  • §14 requires API, SDK and CLI support for a capability to land in one change set — the same rule that put rules favorite in this PR.

Lands in: polyswarm-cli
Proposed fix (untested): add -i/--livescan-id to live feed, passed only when set. Guard it the way rules favorite already does — a clean upgrade message when the installed SDK's live_feed has no such parameter, never a TypeError traceback. That is deliberately the rules favorite precedent and not the --include-counts machinery the previous round removed: the objection there was that the flag crashed on the declared floor, not that a new surface may require the newer SDK.

Give the four rules list filters the same treatment. Two guard sites justify a small helper in client/utils.py rather than repeating the check.

Update the live and rules rows in specs/02-commands.md, and the floor-exception paragraph in specs/05-sdk-contract.md — it currently names rules favorite as the one command exceeding the published floor, and after this set that is three surfaces on two commands.

[MODERATE] F5. live feed asks for 24 minutes beside a badge that counts 24 hours

What happens: A user reads New live results (last 24h): 12 on a ruleset, runs polyswarm live feed, and sees far fewer than 12 with no error. The default asks the server for the last 1440 seconds — 24 minutes — while the badge beside it counts 24 hours.

When: Every live feed run that does not pass --since.

Why:

  • src/polyswarm/client/live.py:34 sets default=1440 with help text "How far back in seconds"; the server converts with timedelta(seconds=since).
  • 1440 is 24 × 60 — a day encoded in minutes, written in 2022 against an SDK whose live_feed docstring said "minutes" until the paired PR corrected it.
  • The value was always right and the unit under it was wrong. The window was intended to be 24 h from the start, so the agreed resolution is a wire fix — minutes becomes the unit everywhere — rather than a relabel here.
  • The new line that makes the two surfaces disagree is src/polyswarm/formatters/text.py:313.

Pre-existing, exposed here
Lands in: polyswarm-cli · Also touches: the internal API's live-feed view, and live_feed's docstring in polyswarm/polyswarm-api#321
Proposed fix (untested), the part that lands here:

  1. --since keeps default=1440 — under minutes that is 24 h, which is the window it was always meant to be. Correct the help text to name minutes.
  2. Add --max-results with no default, so nothing any current invocation returns changes. Implement it as a truncation of the SDK generator (for i, result in enumerate(api.live_feed(...)): if max_results and i >= max_results: break). It rides F4's paired-SDK guard on the same command, so it needs no machinery of its own; if you also pass it through to the SDK's new max_results kwarg to size the page request, that is what the guard is already there for.
  3. Update the live row in specs/02-commands.md.

Naming: --max-results rather than --limit, both to avoid colliding with the server's limit (which means page size, not a result cap) and because it says what it does.

Cassettes need no re-recording. The three live feed recordings pin ?since=9999999, which is "everything" under either unit, so the request is byte-identical and the replayed response unchanged.

Why --max-results is worth adding even though nothing here is unbounded by default: the server has never run an unbounded query — its paginator caps every request at 50 rows, max 1000. The SDK's _paginate, however, follows cursors up to _MAX_PAGES = 10_000. That was unreachable while the default meant 24 minutes; it becomes reachable now the same default means 24 hours, and outright for anyone passing --since 0 for the everything-feed. --max-results is the control for that, not a safety requirement.

  • [LOW] F10. The badge's 24 h product window is stated in three repos with no server-side declaration — the refresh job's window flag, the chart's args, and this repo's hardcoded New live results (last 24h) label (src/polyswarm/formatters/text.py:313). The payload carries new_results_counted_at but no window field, so the client cannot render it from data; tuning the job silently makes this label lie. Fix (untested): lands in the deployment chart as a comment naming the client labels that duplicate the window, so a change there is known to require a paired change here. Nothing to do in this repo unless the window actually moves.

elsewhere: F3, F8, F9 → polyswarm/polyswarm-api#321 · F1, F2, F7 → the deployment chart PR · F6 → the internal API PR

Outstanding review feedback

Status Raised The ask Disposition
open — not a defect both public repos The branch name carries an internal ticket id into public history. The cross-repo CI seam matches on branch name, so it cannot be renamed now; a squash merge with a clean subject satisfies both and needs no rebase. Nothing in the diff can settle it — it is a merge-time action. —

Everything else raised here is addressed. Verified rather than taken on report: the floor guard split into _needs_favorite_method / _needs_favorite_resource, the JSON leg and a real FAVORITE_LIMIT 400 both recorded, specs/05's imports table carrying RequestException/.request.errors, and the pin header corrected from >=4.2.0 to >=4.3.0. The re-recorded ruleset cassettes are genuine — real timestamps, the tracking keys present, livescan_id a digit string, and the favorite counters internally consistent across the toggle.

Standards conformity

The project-level audit stands from round 1. This round introduces no new change-level violation here.

Set-level rows, which belong to every member:

  • §14 delivery order — the set is one capability (API + SDK + CLI + deploy) with the UI legitimately following. Coverage is incomplete, and this repo is where the gap sits: the feed scope and the list filters have no CLI leg. → F4
  • Rule 6 name identity — clean. All four branches are byte-identical, so the harness resolved every sibling at the matching image tag and CI exercised the change together.
  • ## Requires linkage — present on all four, but two entries misdescribe the deploy member. → F2

sbneto added 3 commits August 27, 2026 21:30
A new OPTION may require the newer SDK; an existing INVOCATION may not. Passing
an unknown keyword to an older SDK raises a bare TypeError that
ExceptionHandlingGroup renders as a traceback plus 'Please contact support', so
the guard fires only when the caller actually uses the new option and produces
a clean upgrade message instead.

Inspects the installed signature rather than catching TypeError, so a genuine
argument error inside the SDK is never mistaken for a version mismatch. Two
commands need it, which is what earns a helper over an inline check.
rules view renders a per-ruleset new-results badge and there was no way to list
the results it counts: live feed gains --livescan-id, the badge's drill-down.
It also gains --max-results, and rules list gains the four server-side filters
the SDK exposes (--name, --status, --favorites-only, --has-new-results). The
list is keyset-paginated, so filtering locally would mean walking every page.

--since is documented in MINUTES to match the corrected wire unit; its 1440
default is unchanged and now means the 24h it was always meant to mean. 0 means
no time filter at all.

Every new option is forwarded only when passed, so an unfiltered rules list and
a plain live feed still reach the floor SDK's own signatures untouched. specs/05
records all three floor-exceeding surfaces and why the floor itself does not
move.
Covers the three decisions the plumbing now makes: an unfiltered list still
calls a zero-argument ruleset_list(), a filtered one forwards exactly the
filters given (a False flag must not become favorites_only=False, a filter the
caller never asked for), and live feed forwards the two new kwargs only when
passed. Each new surface is also pinned to degrade with a clean message at exit
2 against a stand-in carrying the floor signature, never a traceback.
@claude

claude Bot commented Aug 28, 2026

Copy link
Copy Markdown

Reviewed against AGENTS.md + specs/02, specs/03, specs/04, specs/05. Gitflow is fine (base develop, no pyproject.toml version bump, SDK PR linked under ## Requires). Five things worth acting on, ordered by severity.


1. formatter_hunt_fields_test.py — the floor guards are applied to the wrong tests, so the suite is not "honest on both installs"

The file docstring says "These tests must stay honest on BOTH installs", and _needs_favorite_method / _needs_favorite_resource are described as "deliberately as NARROW as each dependency". But the narrowness reasoning stops at the class level and misses the attribute-level dependency: _ruleset() / _hunt() instantiate resources.YaraRuleset / resources.HistoricalHunt, which do exist on the pin's floor (4.3.0) — they just do not parse favorite, rule_count, historical_hunt_count, new_results_count, rule_id, rule_modified, source_rule_changed. Resources declare attributes explicitly (that is the premise of every getattr guard in this PR, and of specs/03 §Known-good), so on a floor install the guards silently return None and these tests fail rather than skip:

  • test_ruleset_tracking_fields_render_with_zero_distinct_from_absent
  • test_ruleset_staleness_marker_renders_beside_the_count
  • test_hunt_provenance_fields_render_with_the_reference_point

test_ruleset_none_and_false_fields_are_omitted is worse than a failure — on the floor it passes vacuously, asserting only absence, so it looks like coverage while pinning nothing. Guard these the same way, on the attribute rather than the class (e.g. skipUnless(hasattr(_ruleset(favorite=True), 'favorite'))), or state in the docstring that the formatter fixtures require the paired SDK unconditionally.

2. No end-to-end coverage for the hunt-provenance lines or the new-results badge, despite the PR body claiming the cassettes carry them

The body says "All hunt-shaped VCR cassettes are re-recorded … every ruleset body carries the tracking keys; list bodies carry the stored counter pair". What the recordings actually contain:

  • test_historical_hunt_list_text.vcr: rule_id: null, rule_modified: null, source_rule_changed: null
  • test_ruleset_list_json.vcr: new_results_count: null, new_results_counted_at: null
  • test_ruleset_view_json.vcr: no new_results_count / new_results_counted_at key at all

So four of the new text legs — Source Ruleset Id, Source ruleset last modified at freeze, Source ruleset changed since this hunt froze it, New live results (last 24h) + New-results count refreshed at — never render in any .click expectation. Only the Style-3 unit tests touch them, which specs/04 allows, but the description should not claim cassette coverage that is not there. Concretely missing: a cassette over a ruleset whose stored badge is non-null, and one over a historical hunt frozen from a ruleset modified since (source_rule_changed: true) — the tri-state true branch and the staleness marker are exactly the legs a hand-written fixture cannot pin against a server rename.

3. live feed --since silently changes documented units, and now contradicts its sibling command

client/live.py:36-38 re-documents --since as MINUTES (was "seconds"), with no code change and no test. client/historical.py:66 still reads 'How far back in seconds to request results.' for the same SDK since parameter. polyswarm.py:218 (download_stream) documents minutes. One of live/historical is wrong — fix both in this PR, or say which the SDK actually means. The new 'Pass 0 for no time filter at all' claim also has no test and no cassette; test_plain_feed_forwards_neither_new_kwarg only pins the 1440 default.

This is a user-facing semantics claim now baked into specs/02-commands.md, so it should not be an unverified drive-by.

4. Spec drift — specs/05-sdk-contract.md import table

The new row reads:

from polyswarm_api import exceptions (bare, not aliased) | RequestException — caught by rules favorite…

client/rules.py:3 actually does from polyswarm_api import exceptions as api_exceptions — the aliased form the row directly above already covers. Either fold RequestException into the existing api_exceptions row or drop the "(bare, not aliased)" claim; as written the spec describes an import that does not exist in the code, which is the drift the spec convention exists to prevent.

5. PR description does not describe the diff

The body's "What's new" says "rules list is zero-argument again" while the diff adds four filters to it (--name, --status, --favorites-only, --has-new-results), and never mentions live feed --livescan-id / --max-results or the --since unit change at all — those surface only in the specs/ diff. Given the paired-SDK story turns on exactly which surfaces exceed the floor, the description should list all three guarded surfaces the way specs/05 does.


Minor: the newer than 4.3.0 floor string is hardcoded in three places (client/utils.py:47, client/rules.py upgrade message, and the guard's own text). When the follow-up floor bump lands (specs/05 §Current floor), one is easy to miss — worth a single module constant.

The guard design itself (require_sdk_kwargs inspecting the installed signature rather than catching TypeError, getattr on the method for favorite, forwarding new kwargs only when passed) is right and matches specs/05, and the FAVORITE_LIMIT → CLI PolyswarmException → exit 2 path checks out against ExceptionHandlingGroup in client/polyswarm.py:134-160.

sbneto added 2 commits August 27, 2026 21:48
The floor version was hardcoded in three places, so the follow-up bump that
drops the guards had three chances to miss one. SDK_FLOOR states it once.

--max-results takes IntRange(min=0): 0 is meaningful (no bound, matching the
SDK) but a negative is a typo, and refusing it at the interface beats silently
treating it as unbounded.
…lass

The floor guards were keyed on the resource CLASS, but YaraRuleset and
HistoricalHunt exist on the published floor — they simply do not parse the
tracking and provenance keys. Resources declare attributes explicitly, so on a
floor install the formatter's getattr guards return None and these tests FAIL
rather than skip. test_ruleset_none_and_false_fields_are_omitted was worse: it
asserts absence, so it passed vacuously there, looking like coverage while
pinning nothing. Keyed on the attribute the fixture actually needs.

specs/05's import row also claimed a bare `from polyswarm_api import exceptions`
while rules.py aliases it — the drift the spec convention exists to prevent.
@claude

claude Bot commented Aug 28, 2026

Copy link
Copy Markdown

Reviewed against AGENTS.md and specs/01, 02, 03, 04, 05. Gitflow is clean: base is develop, no CLI version bump, ## Requires links the SDK PR, and specs/02 + specs/03 + specs/05 were updated in the same PR. Five things worth acting on.

1. The floor is triplicated, contradicting the invariant this PR just wrote (src/polyswarm/client/utils.py:19)

specs/05-sdk-contract.md:80 (added here) says the drift "is why the floor lives in ONE authoritative place, the pin, and this doc must follow it". But the floor now exists in three places that nothing ties together:

  • pyproject.toml:25 — polyswarm_api>=4.3.0
  • src/polyswarm/client/utils.py:19 — SDK_FLOOR = '4.3.0'
  • four hardcoded literals in tests (formatter_hunt_fields_test.py:241,289,367, cli_test.py:25)

The comment on SDK_FLOOR says it is "named once so the follow-up bump… has a single place to look" — but the follow-up bump touches pyproject.toml, and nothing fails if SDK_FLOOR stays at 4.3.0. Every user-facing guard message would then name the wrong version, and the tests (which assert 'newer than 4.3.0') would still pass. Easy to miss because src/polyswarm/__init__.py:1 is also 4.3.0 — the CLI's own version and the SDK floor coincide today.

Either derive it (parse the specifier off the installed distribution metadata) or add a test asserting SDK_FLOOR equals the pin's lower bound, and have the tests assert against utils.SDK_FLOOR rather than the literal.

2. specs/03-formatters.md:144-146 still declares the floor as 4.2.0

"Both attributes ship in SDK 4.1.0, but the dependency floor is polyswarm_api>=4.2.0"

Stale against pyproject.toml (>=4.3.0) and against specs/05 §Current floor. This PR edits this exact file (+21 lines), and specs/05 §Current floor calls out this precise drift as the reason the pin is authoritative — fix it here.

3. --max-results 0 fires the version guard for behaviour documented as unchanged (src/polyswarm/client/live.py:42-69)

Help text: "Unset or 0 means no bound — every page, as before." Code:

if max_results is not None:
    kwargs['max_results'] = max_results

So live feed --max-results 0 puts max_results=0 in kwargs and trips require_sdk_kwargs. On the pin's floor, a caller asking for exactly the pre-existing unbounded behaviour gets "live feed --max-results requires a polyswarm-api release newer than 4.3.0". That contradicts the rule specs/05 §Current floor states as the reason the floor does not move — "a new option may require the newer SDK, but an existing invocation may not". if max_results: makes the code match its own help text and keeps 0 off the wire entirely.

Related missing case: no test covers --max-results 0. LiveFeedOptionsTest covers unset and 5 only, so nothing pins that 0 means unbounded on either side of the boundary.

4. FAVORITE_LIMIT message can render (None of None used) (src/polyswarm/client/rules.py:102-106)

errors.get('favorites_used') / errors.get('favorites_limit') are interpolated unguarded, while the server's own human-readable result string ("Favorite limit reached (5 of 5 used).", present in tests/vcr/test_ruleset_favorite_limit_text.vcr:25) is discarded. An envelope carrying code but not the counters yields "Favorite limit reached (None of None used)". Guard the counters, or fall back to the server's result string.

5. Ticket ID in the branch name will land in the merge commit

AGENTS.md §Commit + PR hygiene: "Don't reference ticket IDs or internal project codes in commit messages, PR titles, or PR descriptions." The commits and PR title/body are clean, but the branch is DN-8480-hunting-schema-migration, which a merge commit puts into develop's history as "Merge pull request #266 from polyswarm/DN-8480-…". Squash-merge with a clean subject.


Minor, no action needed: the re-recorded hunt cassettes all carry rule_id / rule_modified / source_rule_changed as null, so the populated provenance render has unit coverage (test_hunt_provenance_fields_render_with_the_reference_point) but no e2e coverage — fine per specs/04 Style 3. Just noting that the PR body's "re-recorded against a stack running the paired server branch" is only load-bearing for the ruleset bodies.

--max-results 0 is documented as the pre-existing unbounded behaviour, but it
was forwarded to the SDK and therefore tripped the floor guard: a caller asking
for exactly what the floor already does got 'requires a polyswarm-api release
newer than 4.3.0'. That contradicts the rule specs/05 states as the reason the
floor does not move — a new OPTION may require the newer SDK, an existing
INVOCATION may not. 0 now stays out of kwargs and off the wire entirely.

The FAVORITE_LIMIT counters are advisory and an envelope can carry the code
without them; interpolating them unguarded rendered 'Favorite limit reached
(None of None used).' at the user. The server's own message is the fallback.

SDK_FLOOR is now tied to the pin by a test — it existed only so the guard
messages could name the floor, and nothing failed if it drifted from
pyproject.toml while every message named the wrong version. The guard tests
assert against the constant rather than a literal. specs/03-formatters.md still
declared the 4.2.0 floor, which is the drift specs/05 names the pin to prevent.
@claude

claude Bot commented Aug 28, 2026

Copy link
Copy Markdown

Reviewed against AGENTS.md and specs/02, 03, 05. Gitflow is right (base develop, no pyproject.toml version bump), the floor-guard design matches what specs/05 §Current floor now documents, and the formatter legs follow the getattr convention in specs/03. Six things worth acting on.

1. --livescan-id help points at a command that never renders the badge. src/polyswarm/client/live.py:56-58 (and the same sentence in specs/02-commands.md) says the badge is "the per-ruleset new-results badge that rules view renders". rules view calls ruleset_get, and the re-recorded detail cassette has no such key — grep -c new_results_count tests/vcr/test_ruleset_view_json.vcr is 0, tests/vcr/test_ruleset_list_json.vcr is 1. Commit b430a14 says this is deliberate ("the detail serializer deliberately does not render the badge"). So the docstring a user reads in live feed --help sends them to the one ruleset command that never shows the number. Should name rules list, in both the docstring and specs/02.

2. exc.request.result is accessed unguarded inside the handler that exists to avoid tracebacks. src/polyswarm/client/rules.py:108. .errors two lines above is read through getattr(..., None), but .result is not. If the SDK request object does not carry .result (or the envelope omitted it), this raises AttributeError inside the except block and the user gets exactly the traceback + "Please contact support" the branch is written to prevent. The only test covering this path (tests/formatter_hunt_fields_test.py:387) builds request as a bare mock.Mock() and assigns request.result itself, so a Mock has the attribute no matter what the real class does — the test cannot fail on this. Use getattr(exc.request, "result", None) or "Favorite limit reached." (or fall back to str(exc)). Note the counters-present path is cassette-pinned, so only the fallback is unpinned — which is why it needs the guard.

3. test_favorite_limit_without_counters_uses_the_server_message is missing @_needs_favorite_method. tests/formatter_hunt_fields_test.py:387 patches polyswarm_api.api.PolyswarmAPI.ruleset_favorite with autospec=True and no create=True. On a floor install (published 4.3.0, no such method) mock.patch raises AttributeError and the test errors instead of skipping — every sibling in that class is either guarded or uses create=True. The file's own header says "Every test touching the new surface is guarded on the narrowest dependency it actually needs"; this one slipped.

4. Two of the four new rules list filters are never signature-checked. test_filters_are_forwarded_only_when_given (tests/formatter_hunt_fields_test.py:210) exercises only --name and --favorites-only. The autospec'd mock is what turns these assertions into a signature check against the installed SDK, so status and has_new_results currently have no check that the CLI's kwarg names match the SDK's. A rename on either side would not fail CI — it would ship as require_sdk_kwargs refusing with "requires a polyswarm-api release newer than 4.3.0" on an SDK that actually has the surface, which is the most confusing possible failure. Adding --status active --has-new-results to that same invocation covers it.

5. specs/05-sdk-contract.md imports table now has two rows with an identical left column (from polyswarm_api import exceptions as api_exceptions, lines 21-22). Fold the RequestException / .request.errors detail into the existing row — as written the table reads as two different imports.

6. The branch name carries an internal ticket ID. AGENTS.md: "Don't reference ticket IDs or internal project codes in commit messages, PR titles, or PR descriptions." Title and body are clean, but a merge commit generated from this branch embeds the ID in the public history. Squash-merge with an explicit title, or rename the branch.

@claude

claude Bot commented Aug 31, 2026

Copy link
Copy Markdown

Reviewed against AGENTS.md and specs/01–05. No correctness defects found. Checked and confirmed: base is develop; CLI version untouched (only the SDK floor moved, which is what specs/05 §Current floor now prescribes); the paired SDK PR exists on the same branch name and declares a clean 4.4.0, so .gitlab-ci.yml's $CI_COMMIT_BRANCH.zip resolves and the unpublished floor is satisfiable in CI; exc.request.errors, exc.request.json['result'] and RequestException(request, *args) -> PolyswarmException all match the SDK source, so the FAVORITE_LIMIT handler and the exit-2 ordering claim in specs/01 hold; no getattr/hasattr probes survive; the un-re-recorded cassettes are unaffected because every new render leg is is not None-guarded.

Three findings, all docs/low severity.


1. specs/04-testing.md contradicts itself on why the FAVORITE_LIMIT cassette must exist — and the answer decides whether the new AGENTS.md carve-out is needed at all.

Line 12 justifies covering the same path twice with: "dropping the cassette would leave the CLI asserting a shape nothing checks." Line 16 then says: "the SDK's own respx suite pins the envelope shape." Both cannot be true. polyswarm-api#321's test_favorite_limit_refusal_is_machine_readable does pin request.errors['code'] plus both counters, so line 16 reads as the accurate one — which collapses line 12's rationale, and with it the reason to weaken the previously-absolute AGENTS.md invariant (the suite must pass against a live e2e stack with VCR off) for one test.

Either drop test_ruleset_favorite_limit_text and revert the AGENTS.md/specs/04 carve-outs (coverage retained: the SDK-boundary mock keeps the message + exit code, the SDK respx suite keeps the envelope), or state plainly that the cassette pins the CLI↔SDK seam and delete the "nothing checks" claim. Right now the spec argues both sides.

2. The stated VCR-off failure mode for that test is wrong, which makes the §Re-recording instruction misleading.

specs/04 line 15: "a VCR-off run gets a 200 and the assertions fail." It won't — ruleset 45874884769561543 does not exist on a live stack, so a VCR-off run 404s and exits 1. Same conclusion, wrong mechanism. Worth fixing because §Re-recording now instructs the next person to saturate the team's five favorite slots before recording, and that is not in fact what separates this test from its siblings. (The real separator: every id-bearing cassette test already depends on recording-stack ids, so the "must pass with VCR off" invariant was aspirational before this PR — the new §exception implies it was literal.)

3. Untested branch: the --unfavorite path through the FAVORITE_LIMIT handler.

client/rules.py:101 — remedy = '' if unfavorite else .... Every test in RulesFavoriteCommandTest that reaches the limit branch invokes without --unfavorite, so the empty-remedy arm never executes and the assert '--unfavorite' in result.output assertions would still pass if the conditional were inverted. One rules favorite 5 --unfavorite case with the same FAVORITE_LIMIT side effect, asserting the remedy sentence is absent, closes it.


Also noted, no action implied: the --since default widening (24 min -> 24 h) combined with an unset --max-results means a bare live feed now pages a full day. It's argued in the PR body, in specs/02, and in --help, and the repo has no CHANGELOG — so the release-note obligation lands squarely on the develop -> master PR.

sbneto added 2 commits August 31, 2026 16:31
…st that cannot meet it"

The carve-out was wrong three ways, and reverting is cheaper than repairing it.

Wrong on the count. It named ONE test that cannot pass with VCR off. Three other
cassettes added here pin server-generated values a live stack will not reproduce
(`Favorited at: 2026-08-26 ...`, `Favorites used: 2 of 5`), because .click
snapshots are exact string equality. So is a cassette already on develop, which
pins `Created at: 2022-05-26 ...`. The next contributor would have counted wrong.

Wrong on the mechanism. It said a VCR-off run "gets a 200 and the assertions
fail". The recorded ruleset id does not exist on a fresh stack, so such a run
404s and exits 1. Same conclusion, wrong reason — and the re-recording steps
were written around that wrong reason.

Wrong on the venue. The condition it described is not something this change
introduces: exact-match cassettes have never been runnable against a fresh
stack. Weakening a project-wide invariant to accommodate a pre-existing
condition is a maintainer decision, not a line item in a feature PR.

AGENTS.md goes back to develop's wording verbatim. The tension between the
invariant and snapshot-style cassettes is real and stays on the record as-is,
for a decision made on its own terms.

Also drops the claim that dropping the cassette "would leave the CLI asserting a
shape nothing checks" — the paired SDK's respx suite does check it. What the
cassette actually pins is the seam between the two, which is what it now says.
…igit count

The envelope-spelling test assigned request.json itself, so it could not detect
the rename it was written to detect — its own comment explains that a real
request was chosen over a Mock because "a Mock fabricates whatever attribute it
is asked for", and assigning the attribute does the same thing. It now pins the
spelling on the untouched request first, before assigning: .json present, the
mistaken .result spelling absent. Verified by renaming the attribute across the
SDK (the class annotation as well as the assignments) and watching it fail.

The empty-remedy arm of the FAVORITE_LIMIT handler had never executed: every
test reaching that branch invoked without --unfavorite, so inverting the
conditional would not have failed one. Verified by inverting it. Now covered.

The known-good getattr guard carried the rationale this branch spent its
commits removing — "a CLI running against an older installed SDK won't have
them". Those attributes ship three releases below the floor, so the pin already
guarantees them; the guard survives as belt-and-braces for an unsupported
configuration, which is what the comment now says. Left as-is it invites both
mistakes: deleting the guard, or adding siblings for versions the floor covers.

--livescan-id's help promised "a 17-digit number"; this repo's own cassettes
carry 16-digit ids.

specs/02 records that a negative --since is now refused at parse time rather
than forwarded, and that exit 2 does not identify a server refusal — click uses
it for usage errors too. specs/05's cutoff gains the release-note obligation for
behaviour changes to existing invocations, since there is no CHANGELOG to carry
them.
@sbneto

sbneto commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Three rounds landed together and converged on the VCR carve-out from three different angles. They were
right and it is reverted — AGENTS.md is back to develop's wording verbatim, byte for byte.

Checking the count claim settled it. The carve-out named one test that cannot pass with VCR off; three
other cassettes added here pin server-generated values (Favorited at: 2026-08-26 …, Favorites used: 2 of 5), and develop's own test_live_hunt_start_text.click already pins Created at: 2022-05-26 ….
.click snapshots are exact string equality, so none of these were ever green against a fresh stack. The
stated mechanism was wrong too — that ruleset id does not exist on a live stack, so such a run 404s and
exits 1 rather than getting a 200.

That makes the condition pre-existing, which settles the venue question: weakening a project-wide
invariant to accommodate something this change did not introduce is a maintainer decision on its own
terms, not a line in a feature PR. The tension is real and stays on the record unamended. Also dropped
the "nothing checks that shape" claim — the SDK's respx suite does check it; what the cassette pins is
the seam.

Fixed:

Finding Resolution
request.json assigned by the test that exists to detect its rename Pins .json present and .result absent on the untouched request first. Verified by renaming it across the SDK — class annotation included — and watching it fail. An incomplete rename does not trip it, because the class annotation alone keeps hasattr true.
--unfavorite arm of the limit handler never executed Covered. Verified by inverting the conditional.
text.py guard kept the older-SDK rationale this branch removed Re-worded to belt-and-braces for an unsupported configuration.
--livescan-id help promised "a 17-digit number" Dropped; this repo's cassettes carry 16-digit ids.
exit 2 reads as identifying a server refusal specs/02 now notes click uses 2 for usage errors too.
negative --since guard unrecorded Added beside the default change.

Two need a person, not a commit:

  1. The --since widening. Raised in all three rounds. specs/05's cutoff now carries the standing
    obligation — behaviour changes to existing invocations get listed on the develop → master PR, and
    the release is at least a minor — but the note itself has to be written at release time.
  2. The invariant vs. snapshot cassettes. Pre-existing and unresolved by design here.

One correction to round 3: test_favorite_limit_refusal_is_a_clean_message_at_exit_2 was described as
covering the ground with VCR off. It patches at the SDK boundary, so it is unaffected by VCR either way —
which is why it is not a substitute for the cassette rather than a replacement for it.

148 tests pass.

…nd the wrong failure

The section pointed at 'the VCR invariant above' for an exception that no longer
exists there, and repeated the mechanism the revert corrected: a fresh-stack
recording does not yield a 200. The recorded ruleset id is not on a fresh stack
at all, so that run 404s; the 200 is what a stack WITH the ruleset and an
unsaturated budget returns. Both states have to be set up, and saturating the
budget takes slots the sibling favorite tests need.
@claude

claude Bot commented Aug 31, 2026

Copy link
Copy Markdown

Review — against AGENTS.md + specs/

Code is sound where I could check it statically: exit 2 for the FAVORITE_LIMIT path holds (exceptions.PolyswarmException → third except clause in ExceptionHandlingGroup), the errors dict/list guard and the getattr(exc.request, 'json', None) or {} fallback are both traceback-free, output.hunt is only ever fed HistoricalHunt (live start/stop go through output.ruleset), the conditional-kwargs blocks match the autospec'd assertions, and the pre-existing live feed cassettes all pass --since 9999999 explicitly so the default change doesn't move any recorded query string. Five things to act on.

1. Three new cassette tests contradict the specs/04 VCR invariant, and the carve-out only covers one of them

specs/04-testing.md still reads, unamended:

VCR is an efficiency cache, not a load-bearing requirement. The suite must pass against a live e2e stack with VCR off. […] if a test only works against its recorded cassette, that's a bug in the test.

The PR adds a note carving out test_ruleset_favorite_limit_text ("needs the team's favorite budget already saturated"), but two more new tests are equally stack-state-bound and are not carved out:

  • tests/vcr/test_ruleset_favorite_text.click pins Favorites used: 2 of 5 — i.e. exactly one other ruleset already starred.
  • tests/vcr/test_ruleset_unfavorite_text.click pins Favorites used: 1 of 5.

_assert_text_result compares the whole rendered block verbatim, so there's no loosening these in place. Either extend the carve-out to name all three (and say what state each needs), or state plainly in the invariant bullet that the ruleset-favorite cassettes are exempt. As written, the spec asserts something the PR's own tests falsify.

2. Confirm the paired SDK actually declares 4.4.0 — every new cassette records 4.3.0

pyproject.toml moves to polyswarm_api>=4.4.0, but the user-agent in all four new cassettes is polyswarm_api/4.3.0:

tests/vcr/test_ruleset_favorite_text.vcr:       user-agent: polyswarm_api/4.3.0 (...)
tests/vcr/test_ruleset_favorite_limit_text.vcr: user-agent: polyswarm_api/4.3.0 (...)

That's circumstantial (the recording box may predate the bump), but specs/05 §Version pin makes it the one precondition for merging this side, and the PR itself spells out the failure mode:

if the archive's declared version is below the floor, that second install silently pulls a newer SDK from PyPI over the archive build […] pip install .[tests] fails

Before merge, read version in pyproject.toml and __version__ in __init__.py off the polyswarm-api branch and confirm both are a clean 4.4.0 with no .dev suffix — the spec's own PEP 440 note (4.4.0.dev0 < 4.4.0) is the trap here. 4.4.0 is not on PyPI, so if the SDK branch is short of it there is nothing to fall back to.

3. --since 0 promises server behaviour nothing verifies

src/polyswarm/client/live.py:39 tells users "Pass 0 for no time filter at all", and the existing cassettes confirm since goes straight into the query (/v3/hunt/live/list?since=9999999&community=gamma) — so --since 0 sends since=0. test_zero_since_IS_forwarded_unlike_zero_max_results pins only that the CLI forwards it, not that the server reads 0 as "unfiltered" rather than "a zero-second window". If that's a server-side guarantee, cite it in specs/02 beside the unit note; if it isn't, the help text is a promise the CLI can't keep.

4. Two omission arms in the ruleset leg are unasserted

formatter_hunt_fields_test.py pins both nested-omission arms for ruleset_favorite (test_unfavorite_carrying_a_stale_timestamp_hides_it) but not the matching arms in ruleset:

  • favorite=True, favorited_at=None → "Favorite: yes" with no "Favorited at" line (text.py:304).
  • new_results_count=3, new_results_counted_at=None → count with no staleness marker. test_ruleset_tracking_fields_render_with_zero_distinct_from_absent builds exactly this state but never asserts 'New-results count refreshed at' not in rendered, so the inner guard at text.py:316 is currently unexercised.

One added negative assertion in the existing test covers the second.

5. Branch name puts a ticket ID into develop's history

Title and description are clean, but the branch is DN-8480-hunting-schema-migration, and a merge commit generated from it writes DN-8480 into the public git log — which is what AGENTS.md §"Commit + PR hygiene" excludes ("Track tickets in the internal tracker, not the git history"). Squash-merge with a clean subject, or rename the branch before merging.

@claude

claude Bot commented Aug 31, 2026

Copy link
Copy Markdown

Reviewed against AGENTS.md and specs/01–05. Code, formatters, tests and all five specs are genuinely in step — the BaseOutput/JSONOutput/TextOutput trio is updated together, ruleset_list() stays zero-arg when unfiltered, generators are iterated, the exit-2 refusal path matches the documented mapping, and the new cassettes are internally consistent (content-lengths match the bodies) rather than copied. Base is develop, pyproject.toml version untouched — gitflow is right. Three things worth a reply before merge.

1. The floor precondition is not evidenced by the recordings (specs/05 §Version pin). Every new cassette records user-agent: polyswarm_api/4.3.0 — tests/vcr/test_ruleset_favorite_text.vcr:20, test_ruleset_unfavorite_text.vcr:20, test_ruleset_favorite_limit_text.vcr:20 — so the SDK build that produced ruleset_favorite was still declaring 4.3.0 when these were recorded, while pyproject.toml now floors at >=4.4.0. Spec 05 is explicit about the failure mode: "if the archive's declared version is below the floor, that second install silently pulls a newer SDK from PyPI over the archive build, and CI stops testing the SDK branch at all". Please confirm polyswarm-api#321's develop head declares a clean 4.4.0 — no .dev suffix — in both pyproject.toml and __init__.py, per §"Version strings must be clean". If it does, this is just a stale UA in a recording and nothing to change.

2. --since 0 = "no time filter at all" is an unpinned promise. src/polyswarm/client/live.py:39 and the command docstring advertise that to users, but test_zero_since_IS_forwarded_unlike_zero_max_results only asserts the 0 reaches the SDK. Nothing establishes that the SDK/server reads since=0 as "no filter" rather than a zero-length window — which would return nothing, i.e. the opposite of what the help says. Contrast --max-results 0, whose meaning is pinned, by the drop. Either cite where that server behaviour comes from (in specs/02, alongside the seconds/minutes note) or soften the help text.

3. historical list --since did not get the same guard. live feed --since gains IntRange(min=0) on the stated grounds that a negative "would be forwarded verbatim to the server", and the new specs/02 note pairs the two options as agreeing on seconds — but src/polyswarm/client/historical.py:66 is still bare click.INT with no lower bound. Apply the guard there too, or narrow the spec note so it does not read as a claim about both.

Not a defect, just the reminder the spec asks for: the live feed default widening (24 min → 24 h) plus unbounded default paging is exactly the kind of change specs/05 §"Behaviour changes to existing invocations are called out at the same cutoff" requires in the develop → master PR. Right now it lives only in this PR body and in specs/02 — and the spec itself says a note that lives only in a spec is not a release note.

…-since 0 rests on

Both nested guards in the ruleset leg had only their True arm exercised. The
tracking-fields test builds a count with no refresh marker — the inner guard's
False arm — but never asserted the marker was absent, and no test rendered a
favorite without a timestamp at all. Inverting either guard failed nothing.
Verified by inverting both; both new assertions fail, and neither did before.

--since 0's help text promises "no time filter at all", which is a claim about
the server, not the CLI. Confirmed against the endpoint: the window is applied
only when since is truthy, so 0 and an absent parameter are the same request.
specs/02 now says so, and says what the CLI test can actually pin — that 0 is
forwarded rather than dropped, which is the half that can regress in this repo.

The re-record steps named one state-dependent cassette. All three
ruleset-favorite cassettes are, because the rendered block is compared verbatim
and the budget counter is inside it: one pins "Favorites used: 2 of 5", another
"1 of 5". They have to be recorded together from a known starting state or the
counters contradict each other.
@sbneto

sbneto commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Items 1, 3 and 4 fixed in 0ef7a28.

4 — the two omission arms. Correct, and worse than stated: neither nested guard in the ruleset
leg had its False arm exercised. The tracking-fields test builds a count with no refresh marker and never
asserted the marker's absence, and no test rendered a favorite without a timestamp at all. Verified by
inverting both guards — the two new assertions fail, and nothing failed before.

3 — --since 0. Settled rather than softened. It is the API's contract: the endpoint applies the
window only when since is truthy, so 0 and an absent parameter produce the same query. specs/02 now
records that, and records what a test in this repo can actually pin — that 0 is forwarded rather than
dropped, which is the half that can regress here.

1 — the re-record guidance. Extended to all three ruleset-favorite cassettes. The counter is inside
the verbatim-compared block, so test_ruleset_favorite_text needs the team holding exactly two stars and
test_ruleset_unfavorite_text exactly one; they have to be recorded together from a known state or the
counters contradict each other.

Note this is guidance on recording, not an exemption. The invariant bullet stays unamended — an earlier
revision of this PR did weaken it and that was reverted, because the condition is not one this change
introduces: .click snapshots are exact string equality, and develop's own
test_live_hunt_start_text.click already pins Created at: 2022-05-26 …. Changing a project-wide
invariant to fit a pre-existing condition is a maintainer decision on its own terms.

2 — the floor. Confirmed and demonstrated: 4.4.0 clean in pyproject.toml (version and
current_version) and __init__.py, no .dev suffix. Installing the SDK branch editable and then
pip install -e .[tests] resolves >=4.4.0 against that checkout with no PyPI fallback. The 4.3.0
user-agents are pre-bump recordings and inert — the matcher is
[method, scheme, host, port, path, query], so headers take no part in replay.

5 — the branch name. Acknowledged and outstanding by decision, not oversight: the merge subject is
where this gets handled, so squash with a clean subject at merge time.

149 tests pass.

@claude

claude Bot commented Aug 31, 2026

Copy link
Copy Markdown

Review

Code, formatters and spec updates line up well against AGENTS.md and specs/01–05: base is develop, the CLI's own version is untouched (only the SDK floor moves), the new group follows the group→verb→formatter→spec checklist, generators are iterated not indexed, and ruleset_favorite was added to base/json/text in step. Three things need action.

1. test_ruleset_favorite_limit_text cannot pass against a live stack alongside the other three favorite tests

specs/04-testing.md (unchanged by this PR): "VCR is an efficiency cache, not a load-bearing requirement. The suite must pass against a live e2e stack with VCR off … if a test only works against its recorded cassette, that's a bug in the test."

unittest orders RulesetTest methods alphabetically, so with VCR off the four favorite tests run:

order test recorded expectation
1 test_ruleset_favorite_json favorites_used: 1
2 test_ruleset_favorite_limit_text server refuses, 5 of 5 used
3 test_ruleset_favorite_text Favorites used: 2 of 5
4 test_ruleset_unfavorite_text Favorites used: 1 of 5

1 → 3 → 4 are internally coherent (+1, then −1 on the same ruleset). #2 is not satisfiable in the same run: it needs the team budget saturated at run time, and _assert_text_result compares the whole rendered block verbatim, so

  • budget full → update readme #3 gets a 400 and exits 2, failing its expected_return_code=0;
  • budget not full → Feature/since flag #2 gets a 200 and its assert 'Favorite limit reached (5 of 5 used)' in expected fails.

The new specs/04 §Re-recording paragraph names this tension ("saturating the budget consumes slots the other favorite tests use") but doesn't resolve it — it documents a cassette-only test rather than fixing one. The FAVORITE_LIMIT path is already covered at the CLI boundary by RulesFavoriteCommandTest.test_favorite_limit_refusal_is_a_clean_message_at_exit_2 plus the no-counters / bare-request / list-shaped variants; the only thing the cassette adds is where in the envelope the code lives, which specs/05 correctly identifies. Options that keep that value without breaking the invariant: pin the envelope shape in a test that doesn't also depend on the team's live budget, or drop the exact-counter assertions from #3/#4 so a live run can land anywhere. As written, the suite is no longer VCR-off-clean.

2. Confirm the paired SDK branch actually declares 4.4.0 (and without a dev suffix)

Every re-recorded cassette carries user-agent: polyswarm_api/4.3.0 — including the three new favorite ones, which exercise a surface 4.3.0 doesn't publish. So the recording environment had the paired branch's code but its declared version was still 4.3.0. specs/05 §Current floor is explicit about the check this PR now rests on: "Read the declared version off the archive's own tree, and mind pre-release suffixes … Check the version string in the SDK branch's pyproject.toml / __init__.py, not the last release tag." With >=4.4.0 here, if the SDK branch still declares 4.3.0 or 4.4.0.dev0, pip install .[tests] goes to PyPI for a version that doesn't exist and CI fails. The PR body asserts the SDK PR declares 4.4.0 — worth pasting the two version strings read off polyswarm-api's branch, since the cassettes are the only in-repo evidence and they say 4.3.0.

3. Branch name publishes an internal ticket ID

AGENTS.md: "Don't reference ticket IDs or internal project codes in commit messages, PR titles, or PR descriptions. This repo is public." The title and body are clean, but DN-8480-hunting-schema-migration is published on a public repo and lands in the default merge-commit subject. Merge with a message that drops it, or rename the branch.

Checked and fine

  • Exit-code routing: exceptions.PolyswarmException → Exit(2), matched before the ancestry-name transport branch that would otherwise take api_exceptions.RequestException to exit 1. ExitCodeHierarchyTest pins the subclass relation the ordering rests on, and specs/01 now documents that the clause order is load-bearing — good thing to have written down.
  • The --since 0 forwarded / --max-results 0 dropped asymmetry is deliberate, tested in both directions, and IntRange(min=0) refuses negatives at parse time with an exit-2 / "Invalid value" assertion rather than a bare != 0.
  • --since 1440 → 86400 is a real behaviour change to an unchanged command line; correctly kept out of pyproject.toml's version (that's the release step) and flagged for the develop → master release note in both specs/02 and specs/05.
  • rules list sends no kwargs when unfiltered (favorites_only or None), signature-checked via autospec.
  • isinstance(errors, dict), the getattr on .request.json, and the ._result-vs-.json spelling are all pinned against real PolyswarmRequest objects rather than Mocks — the reasoning in test_favorite_limit_without_counters_uses_the_server_message about why a Mock couldn't catch it is exactly right.
  • output.hunt is only reached from historical.py, so the new provenance fields can't hit a resource class that lacks them; provenance is null in every recording and unit-pinned instead, as specs/03 states.
  • specs/02's claims that download stream --since is genuinely minutes (IntRange(1, 2880), default 1440) and historical list --since is seconds both check out against the code.

…atisfiable

The three ruleset-favorite cassettes describe one coherent sequence: star a
ruleset (1 of 5), star another (2 of 5), unstar it (1 of 5). The cap refusal
needs 5 of 5. With VCR off, saturating the budget to record or reproduce the
refusal turns the other two into 400s, failing their expected exit 0; leaving it
unsaturated turns the refusal into a 200. There is no stack state where all four
pass, so the suite could not be run against a live stack at all.

The previous commits tried to write that down instead of resolving it -- first
as an exception to the VCR invariant (reverted), then as re-recording guidance.
Documenting a test that cannot run is not the same as having one.

Removing it costs little now, and less than it would have earlier in this
branch. The message and exit code are pinned at the SDK boundary; the envelope
spelling is pinned against a real PolyswarmRequest, which is the part a cassette
was carrying until this branch added that pin; and the wire shape is pinned by
the SDK's stubbed-transport suite -- which declined to record this same refusal,
for this same reason, in the paired PR. Recording it on one side while stubbing
it on the other was the inconsistency.
@sbneto

sbneto commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Item 1 is right, and the mutual-unsatisfiability framing is what settled it. The cassette is removed
(335b01d).

Verified before acting: the three surviving favorite cassettes describe one coherent sequence — star
(favorites_used: 1), star another (2 of 5), unstar it (1 of 5) — and the cap refusal needs 5 of 5.
Saturate the budget and the other two get 400s against their expected exit 0; leave it unsaturated and the
refusal gets a 200. No stack state satisfies all four, so with VCR off the suite could not run at all.

Two earlier attempts on this PR tried to write that down — first as an exception to the VCR invariant
(reverted), then as re-recording guidance. Documenting a test that cannot run is not the same as having
one, and that was the error.

What tipped the cost/benefit is a change made earlier in this same branch: .json present and .result
absent are now pinned against a real PolyswarmRequest, which is precisely the part the cassette was
carrying. So what remains after removal is the message and exit code at the SDK boundary, the envelope
spelling against a real request object, and the wire shape in the SDK's stubbed-transport suite. That
suite declined to record this same refusal for this same reason ("holding all five team slots, racing
every other run"). Recording it on one side while stubbing it on the other was the real inconsistency.

The invariant bullet stays exactly as develop wrote it, now unviolated by this family.

2 — the floor. 4.4.0 clean in pyproject.toml (both version and current_version) and
__init__.py, no .dev suffix. Beyond reading the strings: installing the SDK branch editable and then
pip install -e .[tests] resolves >=4.4.0 against that checkout with no PyPI fallback. The 4.3.0
user-agents are pre-bump recordings and take no part in replay — the matcher is
[method, scheme, host, port, path, query].

3 — the branch name. Outstanding by decision, not oversight; handled in the merge subject.

148 tests pass.

@claude

claude Bot commented Aug 31, 2026

Copy link
Copy Markdown

Review — against AGENTS.md + specs/.

Code side is clean: SDK surfaces used match PR #321 (ruleset_favorite, ruleset_list(name=,status=,favorites_only=,has_new_results=), live_feed(livescan_id=,max_results=), and the parsed resource attrs); the FAVORITE_LIMIT handler reads the envelope off the documented spelling (exc.request.errors / exc.request.json, with _result private — confirmed against the SDK's own specs/05); NotFoundException/NoResultsException still fall through the bare raise and keep exit 1; formatter methods landed on both JSONOutput and TextOutput per AGENTS.md step 3; generators are iterated; and every re-recorded cassette genuinely carries the new keys (provenance null throughout, so the unit pins are the right call). Gitflow is correct — base develop, pyproject.toml version untouched, only the dependency floor moved, with the SDK-release ordering spelled out.

Four things to action.

1. specs/04-testing.md contradicts itself about the FAVORITE_LIMIT cassette (spec drift).

The new invariant at line 12 asserts a recording that does not exist:

FAVORITE_LIMIT is covered both ways deliberately — the SDK-boundary mock pins the message and the exit code, the recorded 400 pins where in the envelope the machine-readable code actually lives. … The cassette is what pins the seam between them.

Line 52 of the same file says the opposite, and is the true statement:

A refusal at the favorite cap is deliberately not recorded here.

specs/05 agrees with line 52 ("It is pinned without a recording"), and grep -rl FAVORITE_LIMIT tests/ returns only tests/formatter_hunt_fields_test.py — there is no such cassette. As written, line 12 misdescribes this PR and licenses a mock+VCR double-cover on the same path, directly against the invariant it is nested under. Delete or rewrite that paragraph.

2. The live feed --since default change is unrelated scope, and its release note has nowhere to live.

The 1440 -> 86400 correction is right on the merits (the wire has always been seconds; historical list --since agrees), but it is the only user-visible behaviour change in a PR titled "render … fields", and it is the sharpest one: combined with --max-results defaulting to unset, a bare live feed goes from a 24-minute window to paging a full day. Two asks:

  • Split it into its own commit at minimum (AGENTS.md commit hygiene: "small, scoped commits — each one should be independently reviewable"), so it can be reverted without unpicking the formatter work.
  • specs/05 as amended by this very PR says "A note that lives only in a spec is not a release note." Right now the only records of this change are specs/02 and this PR body, both invisible at develop -> master time. Put it somewhere the release PR will pick up.

3. Branch name leaks a ticket ID into public history.

AGENTS.md: "Don't reference ticket IDs or internal project codes in commit messages, PR titles, or PR descriptions. This repo is public." Title and body are clean, but the branch is DN-8480-hunting-schema-migration, and GitHub's default merge-commit subject is Merge pull request #266 from polyswarm/DN-8480-… — which lands the code in git history anyway. Squash-merge with an explicit clean subject.

4. --help and comments carry development archaeology.

live feed's docstring is now the primary --help text and leads with the history of the old bug ("It used to default to 1440, which was written as 24*60 believing the unit was minutes — so the real window was 24 MINUTES"), and --livescan-id's help refers to "the badge", which has no referent for someone reading polyswarm live feed --help. Same pattern in code: the kwargs comment in client/live.py and the FAVORITE_LIMIT block in client/rules.py narrate the review that produced them (which spelling "silently yielded None", which test would fail if a branch were folded) rather than what the code does, and they name test functions that will drift. Keep the migration note (--since 1440 restores the old window) in --help; move the rest to the spec.

Nothing blocking on correctness. (1) is the one I would want fixed before merge, since it is a documented invariant that is now false.

…exists

The previous commit removed the favorite-cap cassette but left specs/04's
"one sanctioned exception" paragraph asserting it — "the recorded 400 pins where
in the envelope the machine-readable code actually lives", "the cassette is what
pins the seam between them". Both describe a file this branch deletes, and the
same paragraph licensed mock-plus-VCR coverage of one path, which is the rule it
was nested under as an exception. Nothing needs the exception now: the refusal is
covered by unit tests on this side and a stubbed transport on the SDK's.

Also moves development history out of user-facing help and out of comments that
had grown into review transcripts. `live feed --help` no longer explains that an
old default was written as 24*60 in the belief that the unit was minutes — it
says what the default is and how to get the old window back. "The badge" had no
referent for someone reading --help and is now named for what it is. The kwargs
and FAVORITE_LIMIT comments state the fact at hand and point at the spec for the
reasoning, instead of naming test functions that will drift.
@sbneto

sbneto commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

1 — correct, and my own miss. Deleting the cassette last commit left specs/04 line 12 still
asserting it. Worse than a stale sentence: that paragraph was a carve-out permitting mock-plus-VCR
coverage of one path — an exception to the rule it was nested under — and its entire justification was the
cassette. With the recording gone the exception has nothing to stand on, so the paragraph is deleted
rather than reworded. specs/04 and specs/05 now say the same thing. Fixed in e1b0830.

4 — taken, and it matches a standing convention here (facts inline, rationale in the spec).
live feed --help no longer opens with how an old default came to be written as 24*60 under the wrong
unit; it states the default and how to get the old window back. "The badge" had no referent for someone
reading --help and is now named for what it is. The kwargs and FAVORITE_LIMIT comments stated the
review that produced them and named test functions that will drift — they now state the fact at hand and
point at the spec.

2 and 3 — outstanding by decision, and both belong to the merge, not to a commit here.

  • Splitting the --since change into its own commit means rewriting pushed history on this branch, which
    has been ruled out for it. Recorded rather than done.
  • The release note genuinely cannot live in this PR — that is the point specs/05 now makes, and why the
    obligation was written into the cutoff steps rather than satisfied here. It gets written on the
    develop → master PR.
  • The branch name reaches public history only through the default merge subject, so squash-merging with a
    clean subject is the fix, at merge time.

148 tests pass.

@claude

claude Bot commented Aug 31, 2026

Copy link
Copy Markdown

Reviewed against AGENTS.md, specs/01–specs/05, and the actual surface in the paired SDK PR (polyswarm/polyswarm-api#321). No correctness findings. Everything the CLI now calls directly exists with the names and semantics it assumes:

  • ruleset_favorite(ruleset_id, favorite=True), ruleset_list(name=, status=, favorites_only=, has_new_results=), live_feed(..., livescan_id=, max_results=) — signatures match, and the autospec-based tests make that a real signature check rather than a naming assumption.
  • since=0 genuinely survives to the wire (SDK core._params drops None, keeps 0), so keeping since out of the conditional-kwargs block in client/live.py:79-83 is load-bearing and correctly tested.
  • HistoricalHunt.rule_id / .rule_modified / .source_rule_changed exist only on HistoricalHunt (+ HistoricalHuntList) — and output.hunt is only ever reached from client/historical.py; live start / live stop render via output.ruleset. So the new unguarded attribute reads in text.py:208-216 cannot AttributeError. Worth stating explicitly, since the floor-not-probe rule leaves no fallback.
  • exceptions.RequestException subclasses PolyswarmException, so the re-raise in client/rules.py lands in ExceptionHandlingGroup clause 3 (exit 2) before the MRO-name transport branch (exit 1). ExitCodeHierarchyTest pins the relation the specs/01 note now documents.
  • The SDK branch declares a clean 4.4.0 (no .dev suffix), so >=4.4.0 is satisfiable by the archive CI installs and pip install .[tests] will not silently swap in PyPI.
  • Gitflow: base is develop, ## Requires links the SDK PR, and the CLI own version = "4.3.0" is untouched. Correct.

Three minor items:

1. The json envelope fallback lacks the guard its sibling got — client/rules.py:70

server_msg = (getattr(exc.request, "json", None) or {}).get("result")

errors two lines up is isinstance(...)-guarded specifically so a non-mapping envelope falls through instead of raising on .get (with test_a_list_shaped_errors_envelope_does_not_crash_the_handler pinning it). json gets no such guard, so a non-dict response envelope raises AttributeError inside the handler whose whole purpose is to avoid a traceback — it would fall to the generic except Exception and logger.exception. Narrow (needs a dict errors carrying FAVORITE_LIMIT without counters, plus a non-dict json), but the fix is symmetrical with what is already there: bind the envelope to a local, then envelope.get("result") if isinstance(envelope, dict) else None.

2. The new cassette pair encodes an implicit ordering dependency

test_ruleset_favorite_text pins Favorites used: 2 of 5 and test_ruleset_unfavorite_text pins 1 of 5. That sequence only holds because unittest runs RulesetTest methods alphabetically (favorite_json → favorite_text → unfavorite_text), which specs/04 documents as "the order they run" without saying that the order is the method names. Rename or reorder a method and the sequence breaks — invisibly, because VCR replay still passes; it only surfaces on the next re-record. Naming the mechanism in the new section on cassettes that need a stack state would make that safe.

Related: specs/04:12 still reads absolutely — "if a test only works against its recorded cassette, that is a bug in the test" — while the new section 30 lines below carves out cassettes that need a stack state. Cross-link the bullet to the carve-out so the two do not read as contradicting.

3. Branch name carries a ticket ID

AGENTS.md: "Don not reference ticket IDs or internal project codes in commit messages, PR titles, or PR descriptions." Title, body and all commit subjects are clean — but the head branch is DN-8480-hunting-schema-migration, and a GitHub merge commit renders as Merge pull request #266 from polyswarm/DN-8480-..., writing the ticket ID into this public repo permanent history. Squash-merge with a clean subject (or rename the branch) to keep the rule intent intact.


No action needed on the --since 1440 → 86400 change: it is a real behaviour change to unchanged command lines, and it is handled the way specs/05 requires — documented in specs/02, in the command own --help, and flagged for the develop → master release note.

…y is

`errors` is isinstance-guarded so a non-mapping envelope falls through to the
generic path instead of raising on `.get`. The `.json` read two lines down had
no such guard: `(getattr(...) or {}).get('result')` only substitutes `{}` for a
falsy value, so a list envelope reaches `.get` and raises AttributeError —
inside the handler whose entire purpose is turning this refusal into a clean
message. Reproduced: full traceback plus "Unhandled exception happened. Please
contact support."

Needs three things at once (a dict `errors` carrying the code, no counters, and
a non-dict `.json`), which is why no existing test caught it. One does now;
verified by reverting the guard.

specs/04 also names the mechanism the favorite cassettes' sequence rests on:
unittest orders methods by sorted name, so renaming one breaks the recorded
star/star/unstar sequence silently — replay keeps passing and only the next
re-record surfaces it. And the VCR bullet now points at the re-recording section
rather than appearing to contradict it: the invariant is about a test's logic,
while a .click snapshot pinning server-generated values is a fixture concern.
@sbneto

sbneto commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

1 — real, and reproduced before fixing. errors is isinstance-guarded; .json was not, and
(getattr(...) or {}) only substitutes for a falsy value — so a list envelope reaches .get and raises
AttributeError inside the handler whose whole purpose is avoiding a traceback. Confirmed: full traceback
plus "Unhandled exception happened. Please contact support." Fixed symmetrically, and covered by a test
that fails when the guard is reverted. It needed three conditions at once (dict errors carrying the
code, no counters, non-dict .json), which is why nothing caught it.

2 — taken, both halves. The sequence rests on unittest sorting method names, which the section
described only as "the order they run". Now named, with the consequence: renaming a method breaks the
recorded star/star/unstar sequence silently, because replay keeps passing and only the next re-record
shows it — so renaming one means re-recording all three.

On the cross-link: added, phrased so it points rather than weakens. The distinction that makes them
consistent is that the invariant is about a test's logic — a test whose assertions only hold under
replay is a bad test — whereas a .click snapshot pinning server-generated ids and timestamps is a
fixture property, and true of cassettes long predating this branch. The bullet keeps develop's wording.

3 — outstanding by decision. The branch name reaches public history only through the default merge
subject, so it is handled by squash-merging with a clean subject at merge time.

149 tests pass.

@claude

claude Bot commented Aug 31, 2026

Copy link
Copy Markdown

Review — against AGENTS.md + specs/

Clean against the documented conventions: base is develop, pyproject.toml's own version is untouched (only the dependency floor moves), the SDK PR is linked under ## Requires, text/json formatters stay in sync, the added BaseOutput.ruleset_favorite is additive, generators are iterated rather than materialised, and every spec the change touches (01–05) is updated in the same PR. Exit-code reasoning checks out: exceptions.PolyswarmException is matched at client/polyswarm.py:153, before the ancestry-name transport branch at :164, so FAVORITE_LIMIT really does land on Exit(2) — and ExitCodeHierarchyTest pins the subclass relation that rests on. --since -1 / --max-results -1 do reach the IntRange (click pops the next token as the option value without a dash check), so those parse-time tests assert what they claim.

Four things to action, none in the rendering code:

1. Confirm the paired SDK branch actually declares 4.4.0 before merging. Every new cassette records user-agent: polyswarm_api/4.3.0 (tests/vcr/test_ruleset_favorite_text.vcr:19 and siblings), so the recordings predate the bump this PR's floor names. specs/05 §Current floor makes the merge precondition explicit — the SDK's develop must declare at least the floor, with no .devN suffix. If the paired SDK PR lands without the version bump, CI runs pip install $ARCHIVE/<branch>.zip and then pip install .[tests], and that second install resolves polyswarm_api>=4.4.0 from PyPI over the branch build — exactly the silent-replacement failure specs/05 documents (or a hard failure, since 4.4.0 is not published yet). Replay is unaffected because VCR does not match on headers, so this will not show up as a red test; it has to be checked by hand.

2. The live feed --since change has to reach the develop → master release note, and nothing in-repo carries it there. Two user-visible changes to unchanged command lines: the default window widens 60x (1440 → 86400 s), and a negative --since starts being rejected. The fix is right and specs/02 argues well for why it belongs on this side of the wire — but this repo has no CHANGELOG, and specs/05's own new rule says a note that lives only in a spec is not a release note. Worth carrying forward explicitly, since a bare live feed now pages a full day unbounded.

3. The branch name puts an internal ticket ID into public git history. Commit messages, PR title and PR body are all clean, so the letter of the AGENTS.md rule is met — but the branch ref is just as public and just as permanent, and it is the same leak the rule exists to prevent. Either extend that bullet to cover branch names or drop the prefix next time; not worth renaming mid-PR.

4. Squash on merge. 52 commits, including a feature added and then withdrawn inside the branch (feat: rules list --include-counts, later retracted along with the server aggregate it wrapped), a runtime signature-probe mechanism added and then removed by refactor: pin the SDK floor instead of probing it at runtime, and a literal Revert "docs(specs): …". The final state is coherent; the history is not "independently reviewable" per the hygiene section.

Two notes, no action needed — flagged so they are not surprises later:

  • The favorite cassettes are order-dependent with VCR off: test_ruleset_unfavorite_text.click pins Favorites used: 1 of 5, which only holds if favorite_json and favorite_text ran first in the same session. specs/04 documents this and why the cap-refusal cassette cannot coexist with them, so it is a knowing trade-off — but running one of the three alone against a live stack fails, and a pytest ordering plugin would break it silently.
  • rules favorite under --output-format sha256|sha1|md5 hits BaseOutput.ruleset_favorite's NotImplementedError and reports "contact support". Identical to existing rules view behaviour on those formatters, so pre-existing rather than introduced here.

The hygiene rule names commit messages, PR titles and PR descriptions. Branch
names sit outside it, and reviewers have repeatedly and reasonably read that as
an oversight — a branch ref is just as public and just as permanent.

It isn't an oversight, but the reasoning lived nowhere in this repo. A change
spanning this repo and the SDK has to use the identical branch name in both,
because CI resolves the companion SDK by $CI_COMMIT_BRANCH; for a ticket-tracked
change that name is usually the ticket. So the prefix is deliberate.

What actually leaks is the default merge subject, which interpolates the head
ref. Squash-merging with an explicit subject contains it, and that is now
written down next to the rule it looks like an exception to, along with the
preference for a descriptive shared name where one reads just as well.
@sbneto

sbneto commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

No new code findings this round — all four items are the standing ones, and three of them are decisions
that belong to the merge rather than to a commit here. Recording where each lands.

3 — the branch name. This has come up in every round, so I've written the policy down instead of
acknowledging it again (ac67341). It is not an oversight in the hygiene rule: a change spanning this
repo and the SDK must use the identical branch name in both, because CI resolves the companion SDK by
$CI_COMMIT_BRANCH — for a ticket-tracked change that name is usually the ticket, so the prefix is
deliberate and load-bearing. What actually leaks is the default merge subject, which interpolates the
head ref. AGENTS.md now says so next to the rule it looks like an exception to, along with the
preference for a shared descriptive name where one reads just as well.

4 — squash on merge. Agreed, and it is the same action as item 3: one squash with an explicit clean
subject satisfies both. The withdrawn --include-counts feature, the runtime-probe mechanism that was
added and then replaced by the version pin, and the Revert are all real history of how the branch
converged; squashing is the right way to keep the final state without publishing the path to it.

1 — the floor. Confirmed again: 4.4.0 clean in pyproject.toml (version and current_version)
and __init__.py, no .dev suffix. Beyond reading the strings, the resolution was exercised — installing
the SDK branch editable and then pip install -e .[tests] resolves >=4.4.0 against that checkout with
no PyPI fallback. The 4.3.0 user-agents are pre-bump recordings; agreed they are inert, since the
matcher is [method, scheme, host, port, path, query].

2 — the release note. Cannot be satisfied in this PR by construction, which is why the obligation was
written into specs/05's cutoff steps rather than answered here. It gets written on the
develop → master PR, covering both user-visible changes: the default window widening and the newly
rejected negative --since.

On the two no-action notes: the order-dependence of the favorite cassettes is a knowing trade-off and is
now documented with its mechanism (unittest sorts method names) and its failure mode (renaming one
breaks the sequence silently, since replay keeps passing). The sha256|sha1|md5 formatter behaviour is
pre-existing and identical to rules view, so it is left alone rather than fixed asymmetrically here.

149 tests pass.

@claude

claude Bot commented Aug 31, 2026

Copy link
Copy Markdown

Reviewed against AGENTS.md and specs/01–05. The code side is clean: the SDK surfaces the CLI calls all match polyswarm/polyswarm-api#321 exactly (ruleset_favorite(ruleset_id, favorite=), ruleset_list(name=, status=, favorites_only=, has_new_results=), live_feed(..., livescan_id=, max_results=)), the SDK PR declares a clean 4.4.0 in both pyproject.toml and __init__.py so the floor precondition in specs/05 §Current floor is satisfied, the FAVORITE_LIMIT envelope path (exc.request.errors[...], exc.request.json["result"]) matches what the SDK spec documents as the parsed response body, the exit-2 mapping holds via api_exceptions.PolyswarmException, base is develop, and no CLI version bump. Two things need action.

1. specs/04s favorite re-record recipe does not reproduce the shipped cassettes, and omits an interleaved test.

specs/04-testing.md says the three favorite cassettes "describe one sequence (star, star, unstar)" and must be re-recorded "together, from a known starting state, in the order they run … favorite_json → favorite_text → unfavorite_text". Two problems:

  • test_ruleset_list_json sorts between test_ruleset_favorite_text and test_ruleset_unfavorite_text under unittests alphabetical order. Its snapshot pins a three-ruleset inventory with "favorite": false on all three. Anyone following the recipe records list_json while a ruleset is starred, so that snapshot changes too — it belongs in the re-record set the spec names, or the coupling needs breaking.
  • The shipped cassettes were not produced that way. test_ruleset_list_json.vcr / test_ruleset_view_json.vcr carry date: Tue, 25 Aug 2026 18:26:xx and rulesets 77454540525125655 / 44051669277897879 / 78562964231669682; the favorite cassettes carry date: Wed, 26 Aug 2026 15:01:51 and rulesets 96652060989160147 / 14883307518120680, which do not exist in the Aug-25 inventory at all. Contrast HistoricalHuntTest, which was recorded in one run — test_historical_hunt_list_text.click shows 3220090199138422 as DELETING, the id test_historical_hunt_delete_text deletes just before it. So the spec is describing a workflow that was not used, and following it would surface the inventory contradiction.

Concretely: _assert_text_result compares the whole rendered block, so Favorites used: 2 of 5 / 1 of 5 make favorite_text depend on favorite_json having starred a different ruleset first. _assert_text_result already takes a replace= hook — normalising the counter there ((("Favorites used: 2 of 5", "Favorites used: N of M"),)) removes the cross-test coupling and the whole "re-record in order from a known state" caveat, rather than documenting it.

2. Merge-time: this PR must be squash-merged with an explicit subject.

The new AGENTS.md bullet legalises the ticket-prefixed branch name on the condition that "Squash-merge with an explicit clean subject, and it never lands." The only merge visible in this repos history is Merge pull request #264 from polyswarm/release-4.3.0 — a default merge subject, not a squash. Merged that way, DN-8480-hunting-schema-migration lands in public develop history and the bullet is self-contradicting on its first use. Either squash with a clean subject, or drop the ticket prefix from the branch on both repos (they only need to be identical, not ticket-shaped — as the bullet itself notes).

Nothing else: the --since seconds/86400 correction is consistent with historical list --since and correctly leaves download stream --since on minutes, --max-results 0 vs --since 0 taking opposite paths is pinned in both directions, the getattr carve-out in known_good is documented where it lives, and the formatter/spec pairs are in sync.

… happened

The recipe said to record the three favorite cassettes together, in sorted-name
order, from a known starting state. Two things are wrong with it.

It omits test_ruleset_list_json, which sorts between favorite_text and
unfavorite_text and pins "favorite": false on every ruleset in its inventory.
Anyone following the recipe records it while a ruleset is starred, so that
snapshot changes as well.

And the shipped cassettes were not made that way: favorite_text acts on
96652060989160147, which does not appear anywhere in list_json's inventory
(77454540525125655, 44051669277897879, 78562964231669682). They were recorded
against different stack states, so following the recipe would surface the
contradiction rather than reproduce the fixtures.

Now states the couplings instead of a sequence, keeps the silent-failure warning
about renaming a method, and points at the escape hatch that exists already:
_assert_text_result takes a replace= hook, so normalising the budget counter
makes each cassette independent of what ran before it. Better to break the
coupling than to document a wider one.
@sbneto

sbneto commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Both halves check out, and the recipe is corrected in 6869b2d. Thanks — this one was worth catching.

The interleaved test. Confirmed: sorted-name order is favorite_json → favorite_text →
list_json → unfavorite_text, and test_ruleset_list_json.click pins "favorite": false three times.
Anyone following the recipe records it while a ruleset is starred, so that snapshot moves too.

The provenance. Also confirmed, and more bluntly than stated: test_ruleset_favorite_text acts on
96652060989160147, which appears nowhere in list_json's inventory (77454540525125655,
44051669277897879, 78562964231669682). Those cassettes cannot have come from one pass over one stack
state — so the recipe was describing a run that never happened, and following it would have surfaced the
contradiction instead of reproducing the fixtures.

The section now states the two couplings rather than a sequence, keeps the warning that renaming a method
breaks things silently (replay passes; only the next re-record shows it), and points at the escape hatch
that already exists: _assert_text_result takes a replace= hook, so normalising the budget counter
makes each cassette independent of what ran before it. Documenting a wider coupling is the worse of the
two options, and the spec now says so rather than choosing it.

Not taking the normalisation itself in this PR — it changes what three shipped cassettes assert, and the
coupling is now described accurately enough to act on deliberately.

149 tests pass.

@claude

claude Bot commented Aug 31, 2026

Copy link
Copy Markdown

Review

Checked the CLI diff against AGENTS.md + specs/01–05, and cross-checked every SDK surface it now calls directly against polyswarm-api#321 (ruleset_favorite, the four ruleset_list filters, live_feed(livescan_id=, max_results=), the YaraRuleset/YaraRulesetFavorite/HistoricalHunt attributes, and the FAVORITE_LIMIT envelope spelling request.errors["code"] / request.json["result"]). All shapes line up, the exit-2 path is real (ExceptionHandlingGroup catches exceptions.PolyswarmException before the transport branch — src/polyswarm/client/polyswarm.py:153-161), and Polyswarm overrides neither live_feed nor ruleset_list, so the kwargs fan-out reaches the SDK unmodified. No correctness bugs found.

Three things that need action.

1. source_rule_changed: true — the yes arm is completely uncovered

src/polyswarm/formatters/text.py:214 renders changed = "yes" if result.source_rule_changed else "no". tests/formatter_hunt_fields_test.py pins only False ("…froze it: no") and None (prints nothing), and every recorded cassette carries source_rule_changed: null (17/17). So inverting that ternary passes the entire suite, including the test whose docstring says the label exists so it cannot read as edited recently. That is the exact failure mode the file guards against elsewhere ("Without this the guard could be inverted and nothing would fail" — the new_results_counted_at case). One extra line in test_hunt_provenance_fields_render_with_the_reference_point, or a sibling with source_rule_changed=True.

2. The new cassette set ships a documented inconsistency instead of the fix the spec itself names

specs/04-testing.md adds ~20 lines describing an ordering coupling: _assert_text_result compares verbatim, so test_ruleset_favorite_text pins Favorites used: 2 of 5 and test_ruleset_unfavorite_text pins 1 of 5, test_ruleset_list_json sorts between them and pins "favorite": false on every row — and it states outright that the shipped set is not what a single recording pass produces (the favorited ruleset 96652060989160147 appears in no list cassette). It then names the remedy itself: normalise the budget counter through _assert_text_result(..., replace=), which already exists at tests/cli_test.py:66.

Please apply that rather than document the landmine. AGENTS.md forbids hand-editing cassettes, so the only way out once this lands is a coordinated three-cassette re-record against a stack in a specific state — and the spec concedes the coupling changes silently under a rename or reorder. Two replace= tuples normalising Favorites used: N of M remove the whole section.

3. Merge mechanics

  • Order is forced and this side cannot go first. Per specs/05 §Current floor, "the SDK's develop must declare at least the floor" to merge here. polyswarm-api#321 is still open, so merging this alone leaves develop asking develop.zip for a version it does not declare and pip install .[tests] fails. The PR body says this; flagging so the merger does not miss it. (Verified #321 declares 4.4.0 with no dev suffix, so the floor is satisfiable the moment it lands.)
  • Must be squash-merged with an explicit subject. The branch carries DN-8480, permitted by the AGENTS.md bullet this PR adds — but only on that bullet's own condition. A default merge subject (Merge pull request #266 from polyswarm/DN-8480-…) lands the ticket ID in public history, which is what the surrounding rule forbids.
  • Base develop OK, CLI version untouched OK, ## Requires present OK, formatter method added to both TextOutput and JSONOutput OK, specs updated in-PR for every area touched OK (the client/utils.py vs utils.py clarification in specs/01 is unrelated but correct).

Non-blocking

  • test_favorite_limit_refusal_is_a_clean_message_at_exit_2 and test_the_unfavorite_direction_gets_no_remedy_sentence build the request as mock.Mock(), which the same file argues against two tests later ("A REAL request, not a Mock: a Mock fabricates whatever attribute it is asked for"). Harmless here — both take the counters path and never touch .json — but core.PolyswarmRequest costs nothing and matches the file's own rule.
  • The conditional forwarding in src/polyswarm/client/live.py is redundant against the SDK: _as_result_bound already maps None/0/negative to "no bound", and livescan_id defaults to None. Fine to keep for the call-shape assertions; just noting it is belt-and-braces, not load-bearing.
  • PR body says the re-recorded list bodies "carry the stored counter pair"; they actually carry new_results_count: null / new_results_counted_at: null, so the badge render legs are unit-pinned only — same as the hunt provenance ones. Description nit, not code.
  • The behaviour change worth the release note on develop -> master is the one the PR already identifies: a bare live feed goes from a 24-minute to a 24-hour window with --max-results unset.

@sbneto
sbneto merged commit 91cc5f1 into develop Aug 31, 2026
2 checks passed
@sbneto
sbneto deleted the DN-8480-hunting-schema-migration branch August 31, 2026 20:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants