Adding tasks based on mutation testing

tasks/10-ratio-gate-boundaries.md

@@ -0,0 +1,31 @@
+# 10 — Ratio gate boundaries and branches
+
+**Priority**: P1 · **Class**: NO_REFACTOR · **Depends on**: — (do it alongside 12; the tests here must pin `SKIP_RATIO_CHECK`)
+
+## Problem
+`should_fetch_torrent` / `_should_fetch_devuan` are the project's core business rule: don't fetch
+a new release until the previous one has reached ratio 1.0. Every line of them runs in tests, but
+mutation testing shows the tests don't pin the outcomes. Only 67% of `should_fetch_torrent` mutants
+are killed, and one branch never runs at all.
+
+## Evidence (mutmut survivors, `fetch_torrents.py`)
+- `:434` and `:374`: `>= 1.0` → `> 1.0` survives. No test has a previous ratio of exactly 1.0.
+- `:432`: `return True  # no previous version` → `False` survives, and coverage shows the line is never executed.
+- `:430`: `v < version` → `v <= version` survives.
+- `:364`: `n != name and DEVUAN_NAME_PATTERN.match(n)` → `or` survives. A non-Devuan ratio can count as the "prior Devuan" ratio.
+- `:386–399`: the `SKIP_RATIO_CHECK`, unknown-distro and unparseable-name branches can all be flipped to `return False` without a test failing.
+- `:416`: the documented rule that a live ratio beats the removed-history ratio for the same name is never asserted.
+
+## Acceptance criteria
+- Prior version ratio exactly `1.0` → fetch (`True`); `0.999` → don't fetch (`False`). Do this for both a standard distro and Devuan.
+- Only *newer* versions in `ratios` (no older one) → `True`. This must execute L432.
+- `ratios` contains the *same* version as `name` only → behavior asserted explicitly. The `<` vs `<=` boundary at L430 must be killed.
+- Devuan: `ratios` mixing a low-ratio non-Devuan torrent with a high-ratio Devuan one → the decision uses only the Devuan ratio.
+- Name in both `ratios` (live) and `removed_history` with different ratios → the live value decides.
+- `SKIP_RATIO_CHECK=true` → `True` even with a low prior ratio. Unknown distro → `True`. Unparseable name of a known distro → `True` (and logs an error).
+- Re-running mutmut on `should_fetch_torrent` / `_should_fetch_devuan`: all the survivors listed above are killed.
+
+## Approach
+1. Add the cases to the existing `ShouldFetchTorrent*` classes in `tests/test_fetch_torrents.py`. Wrap each class in `patch.dict(os.environ, {'SKIP_RATIO_CHECK': 'false'})` in `setUp`, so the developer's shell can't flip them (see task 12).
+2. Each new test should fail when the corresponding mutant is applied by hand. Confirm this for at least the `>=` and L432 cases.
+3. No production change is expected. If a test exposes a real bug, fix it test-first in this task.

tasks/11-get-previous-ratios-tests.md

@@ -0,0 +1,26 @@
+# 11 — Test `get_previous_ratios` in-process
+
+**Priority**: P1 · **Class**: NO_REFACTOR · **Depends on**: —
+
+## Problem
+`get_previous_ratios(log_file)` (`fetch_torrents.py:163–195`) parses the previous run's ratio log.
+Its output is the `ratios` input to the ratio gate (task 10). Only `RatioLogSurvivesRestartTests`
+reaches it, and that runs in a subprocess, so coverage records L164–195 as unexecuted and only the
+happy path is checked. The segment-selection logic is untested: last START wins, START without END,
+and an END before START.
+
+## Acceptance criteria
+In-process tests, each writing a temp log file and calling `ft.get_previous_ratios(path)`:
+- Missing file → `{}`.
+- No markers at all → every `[ratio] name → x.yyy` line in the file is parsed.
+- Two complete START/END segments → only the **last** segment's ratios are returned. An entry that exists only in the first segment must be absent.
+- Last START with no END after it (a crashed run) → everything after that START is parsed.
+- An END that appears before the last START is ignored.
+- Names with internal spaces/padding are stripped. A ratio without a decimal point (`→ 1`) is not matched, so the current regex behavior is pinned.
+- Duplicate names inside a segment → the last one wins.
+- Coverage of L164–195 is 100% in the normal `unittest` run.
+
+## Approach
+Add a `GetPreviousRatiosTests` class next to `RatioLogSurvivesRestartTests`, and keep the subprocess
+test as-is, since it guards the import-time truncation bug. Write the log lines in the real format
+(`2026-08-01 00:00:00,000 INFO: [ratio] … → 0.664`).

tasks/12-test-env-clock-isolation.md

@@ -0,0 +1,21 @@
+# 12 — Isolate tests from env vars and the clock
+
+**Priority**: P1 · **Class**: NO_REFACTOR (test-side only; the production seams are task 19) · **Depends on**: —
+
+## Problem
+Some tests pass only because the developer's shell and the wall clock happen to cooperate:
+- `should_fetch_torrent` reads `SKIP_RATIO_CHECK` directly (`fetch_torrents.py:386`). With `SKIP_RATIO_CHECK=true` exported, the `ShouldFetchTorrent*` tests at `tests/test_fetch_torrents.py:241–385` fail. This is confirmed.
+- `cleanup_*` reads `CLEANUP_*`. `get_rpc_credentials` reads `TRANSMISSION_RPC_*`, which affects the tests around `:1663–1712` and `:1777–1866`.
+- `CleanupRemovesWatchFileTests` builds its history from `date.today()` (`:1855`) while production calls `date.today()` again. A run that crosses midnight shifts the 32-day offset.
+- Temp dirs from `tempfile.mkdtemp` are never removed, in all three test files.
+
+## Acceptance criteria
+- Every test class that runs code reading these env vars patches the environment with `patch.dict(os.environ, {...}, clear=False)` and explicitly sets **or deletes** the keys it depends on. Add a small helper, e.g. `isolated_env(**overrides)`, that removes `SKIP_RATIO_CHECK`, `CLEANUP_*`, `TRANSMISSION_RPC_*` and `FETCH_TORRENTS_*` (except `FETCH_TORRENTS_LOG_DIR`) unless they're overridden.
+- Running the whole suite with `SKIP_RATIO_CHECK=true CLEANUP_MIN_RATIO=5 CLEANUP_SKIP_RATIO_CHECK=true TRANSMISSION_RPC_USERNAME=x TRANSMISSION_RPC_PASSWORD=y` exported is green.
+- The stagnation integration test patches `ft.date` (or uses an injected `today` once task 19 lands) to a fixed date. No `date.today()` remains in `tests/`.
+- Each `mkdtemp` is paired with `self.addCleanup(shutil.rmtree, path, ignore_errors=True)`, or uses `tempfile.TemporaryDirectory`.
+
+## Approach
+1. First, run the suite with the hostile env above and record which tests fail. Those failures are the "red" for this task.
+2. Add the helper and apply it, then re-run with the hostile env until green.
+3. Optional: add a CI step that runs the suite a second time with that hostile env, so regressions are caught.

tasks/13-bdecode-truncated-input.md

@@ -0,0 +1,29 @@
+# 13 — Fix `bdecode` accepting/mis-reporting truncated input
+
+**Priority**: P1 (defect) · **Class**: NO_REFACTOR · **Depends on**: —
+
+## Problem
+`torrent_scrape.bdecode` (`torrent_scrape.py:36–75`) is meant to raise `BencodeError` on malformed
+data. Verified on 84761d3:
+
+| Input | Actual | Expected |
+| --- | --- | --- |
+| `b'4:ab'` (declares 4 bytes, has 2) | returns `(b'ab', 6)`, accepted silently | `BencodeError` |
+| `b'i12'` (no terminating `e`) | `ValueError: subsection not found` (from `bytes.index`) | `BencodeError` |
+| `b'iabce'` (non-numeric int) | `ValueError` from `int()` | `BencodeError` |
+| `b'2x:ab'` (bad length) | `ValueError` from `bytes.index`/`int()` | `BencodeError` |
+
+The silent truncation matters most. A truncated `.torrent` download decodes into a wrong `info` dict,
+so `compute_info_hash` produces the wrong hash, and the tracker scrape then reports "no demand"
+rather than "bad file". `BencodeError` subclasses `ValueError`, so existing callers that catch
+`ValueError` keep working.
+
+## Acceptance criteria
+- Each input in the table raises `BencodeError` with a message naming the problem and its index.
+- A string length that runs past the end of the data raises.
+- Existing round-trip tests (`BdecodeBencodeTests`) stay green.
+- Mutation: the new guard lines are killed.
+
+## Approach
+Add the table above as tests in `BdecodeBencodeTests` and confirm each fails today. Then add a length
+check in the byte-string branch, and wrap the `index`/`int` calls so they re-raise as `BencodeError`.

tasks/14-cleanup-boundaries.md

@@ -0,0 +1,25 @@
+# 14 — Cleanup planning boundary tests
+
+**Priority**: P2 · **Class**: NO_REFACTOR · **Depends on**: 12
+
+## Problem
+The cleanup planners decide what gets **deleted from disk**, and they're pure functions. Mutation
+shows boundaries and loop control that no test pins down: `plan_stagnation_cleanup` 60%,
+`_devuan_sort_key` 60%, `plan_cleanup` 67%, `_old_version_candidates` 68%.
+
+## Evidence (mutmut survivors, `fetch_torrents.py`)
+- `:1140` `plan_cleanup`: `ratio <= min_ratio` boundary. No test has ratio exactly equal to `min_ratio`.
+- `:1305` `plan_stagnation_cleanup`: the ratio-delta comparison isn't tested at exactly `min_ratio_delta`.
+- `:1283` `_stagnation_anchor`: an anchor sample exactly `window_days` old isn't tested.
+- `:1270` / `:1274` `update_ratio_history`: the sample-interval gate and the pruning age aren't tested at their edges.
+- `:1096–1112` `_old_version_candidates` and `:1303`: `continue` → `break` survives. No input list puts an unparseable, Devuan or no-anchor torrent **before** valid candidates.
+- `_devuan_sort_key`: ordering ties on `added_date` aren't exercised.
+
+## Acceptance criteria
+- For each boundary, add a test at the edge value and one on each side of it, with the expected keep/remove asserted per torrent. Assert the actual lists, not only their length.
+- For each `continue`, add a test whose list starts with the item that should be skipped, followed by valid candidates. All the valid candidates must still be planned.
+- Re-running mutmut on these functions: ≥ 90% killed, with any remaining survivors documented as equivalent.
+
+## Approach
+Extend `PlanCleanupTests`, `PlanCleanupDevuanGroupingTests`, `PlanStagnationCleanupTests` and
+`UpdateRatioHistoryTests`. Each test needs one fixed `today`, not `date.today()`.

tasks/15-ci-static-gates.md

@@ -0,0 +1,21 @@
+# 15 — Add ruff + mypy to CI
+
+**Priority**: P2 · **Class**: CI (no test-first) · **Depends on**: —
+
+## Problem
+`.claude/CLAUDE.md` lists `ruff check .` and `mypy …` as project commands, but CI
+(`.github/workflows/docker-image.yml`, `test` job) runs only unittest. Current state on 84761d3:
+- `ruff check .`: **42 errors** (6 auto-fixable).
+- `mypy fetch_torrents.py torrent_scrape.py configure_transmission.py`: **2 errors**, both in `fetch_torrents.py`. One is a missing-stubs note for an import at L14. The other is at L1048, where the `json=` argument to `requests.post` has type `dict[str, Collection[str]]`.
+
+## Acceptance criteria
+- `ruff check .` and the mypy command both exit 0 locally.
+  - Fix the code where the finding is real. Where it's noise, use a targeted `# noqa` / `# type: ignore[code]` with a reason, or a rule exclusion in config.
+  - Missing stubs for `transmission_rpc`: use `ignore_missing_imports` scoped to that module only.
+- Ruff and mypy settings live in config (`pyproject.toml` or `ruff.toml` + `mypy.ini`), so CI and local runs agree.
+- The `test` job installs `requirements-dev.txt` and runs `ruff check .` and mypy before unittest. The build job is still gated on `test`.
+- Don't run `ruff format` as part of this task, so the diff stays reviewable. Add a `ruff format --check` step in a separate PR if it's wanted.
+
+## Notes
+`requirements-dev.txt` doesn't pin versions. Pin `ruff` and `mypy` so new upstream rules don't
+break CI without warning.

tasks/16-dev-tooling-coverage-mutation.md

@@ -0,0 +1,20 @@
+# 16 — Dev tooling: coverage + mutmut in requirements-dev
+
+**Priority**: P2 · **Class**: CI/config · **Depends on**: —
+
+## Problem
+The test-health audit had to install `coverage` and `mutmut` 2.5.1 ad hoc. `.claude/CLAUDE.md`
+mentions `mutmut run` (mutmut<3), but neither tool is in `requirements-dev.txt` and there's no mutmut
+config. That makes the baseline in `tasks/README.md` hard to reproduce.
+
+## Acceptance criteria
+- `requirements-dev.txt` includes pinned `coverage` and `mutmut<3`.
+- Coverage config (`[tool.coverage]` or `.coveragerc`): `branch = true`, `source` = the three modules, omit `tests/`.
+- Mutmut config: `paths_to_mutate` = the three modules, `runner = python -m unittest discover -s tests`, `tests_dir = tests/`.
+- `.gitignore` covers `.mutmut-cache`, `mutants/`, `.coverage`, `htmlcov/`.
+- CI `test` job: run unittest under `coverage run --branch` and print `coverage report`. Report only; don't add a fail-under threshold yet (task 27 sets one).
+- `.claude/CLAUDE.md` Commands section lists the coverage command.
+
+## Notes
+Mutmut doesn't belong in the per-push merge check: the full run takes about 40 minutes on Windows. If
+a scheduled mutation job is wanted, it fits in task 23's scheduled workflow.

tasks/17-daily-run-characterization.md

@@ -0,0 +1,39 @@
+# 17 — Lock in daily-run behavior with a `runpy` test
+
+**Priority**: P2 · **Class**: NO_REFACTOR (test only) · **Depends on**: 12
+
+## Problem
+The `if __name__ == "__main__":` block (`fetch_torrents.py:1351–1457`, about 100 lines) is the whole
+daily run. It's untested, and it's where all the parts get wired together:
+- the ratio log feeds `should_fetch_torrent`
+- `removed_history`
+- the per-distro fetch loop and its success/existing/not-found/failure counting
+- the old-release demand check
+- `log_seed_ratios_via_http`, then the cleanup branches selected by env vars
+- `enforce_disk_usage_limit`
+- the summary line
+
+Before task 18 can refactor this block, a behavior lock-in test has to pin down what it does now.
+
+## Acceptance criteria
+A test class, e.g. `DailyRunCharacterizationTests`, runs the module as `__main__` via
+`runpy.run_path(fetch_torrents.py, run_name='__main__')`. It runs:
+- with `transmission_rpc.Client` stubbed (reusing the `sys.modules` stub)
+- with `requests.get`/`requests.post` patched to canned responses for each distro
+- with the watch dir, log dir and state files in a temp dir
+- with `shutil.disk_usage` patched and the env pinned via the task-12 helper
+
+At minimum it asserts:
+- Distros not in `FETCH_TORRENTS_DISTROS` are never fetched.
+- A candidate whose predecessor has ratio < 1.0 is **not** downloaded, and one at ≥ 1.0 is. The ratio log is pre-written in the temp log dir.
+- An exception in one distro's fetcher doesn't stop the others, and it's counted as a failure.
+- Each cleanup function is called only when its env flag is set.
+- Disk-usage enforcement runs after cleanup, and an exception there is logged, not raised.
+- The summary line reports the right counts.
+
+The tests make no network calls and complete in under 2 s total.
+
+## Notes
+Because the module runs again under `runpy`, its import-time logging setup runs again too. Point
+`FETCH_TORRENTS_LOG_DIR` at the temp dir, and close the handlers in `addCleanup`. If this gets too
+awkward, that's evidence for task 18. Don't change production code here.

tasks/18-extract-run-and-logging.md

@@ -0,0 +1,22 @@
+# 18 — Extract `run()` and `configure_logging()`
+
+**Priority**: P2 · **Class**: REFACTOR_REQUIRED · **Depends on**: 17
+
+## Problem
+There are two blockers to testing `fetch_torrents.py` directly:
+1. **The `__main__` orchestration can't be called.** It reads env vars inline (`:1407–1410`, `:1446`, …) and calls `date.today()`, `time.time()` and the module-level fetchers directly.
+2. **Logging is configured at import time.** It opens files under `log_dir` (`:19` onward), which is why every test file must set `FETCH_TORRENTS_LOG_DIR` *before* `import fetch_torrents` (see CLAUDE.md).
+
+## Acceptance criteria
+- New `run(config, *, client_factory, today, now=time.time, fetchers=DEFAULT_FETCHERS) -> RunSummary`, where `config` is a small dataclass built by `load_config(os.environ)` from every env var `main` reads today.
+- `__main__` becomes `configure_logging(log_dir); run(load_config(os.environ), ...)`.
+- New `configure_logging(log_dir)`. Importing the module no longer opens files or adds handlers, and the `RatioLogSurvivesRestartTests` guarantee is preserved: ratios from the previous run are readable before the ratio handler truncates the file.
+- The task-17 lock-in tests pass **unchanged** against the refactor, then are rewritten to call `run()` directly with fakes.
+- `tests/test_fetch_torrents.py` no longer needs the env-before-import workaround for logging. The `transmission_rpc` stub stays until task 20.
+- Update the CLAUDE.md section "Why the test setup looks the way it does" to match.
+- `fetch_torrents.py` branch coverage ≥ 90%.
+
+## Approach
+Do this in small steps, with the lock-in suite green after each one. Extract `load_config` first,
+then `configure_logging`, then `run`. Follow the existing `plan_cleanup()` vs `cleanup_old_versions()`
+pattern: keep the decisions pure and do the I/O at the edges.

tasks/19-inject-cleanup-seams.md

@@ -0,0 +1,27 @@
+# 19 — Inject `today` / client / config into the `cleanup_*` functions
+
+**Priority**: P2 · **Class**: REFACTOR_REQUIRED · **Depends on**: 17 (can go before or with 18)
+
+## Problem
+The I/O wrappers `cleanup_old_versions` (`:1147`), `enforce_disk_usage_limit` (`:1196`),
+`cleanup_stagnant_torrents` (`:1312`) and `should_fetch_torrent` (`:386`) each reach out for their
+inputs, which makes their tests patch-heavy and fragile:
+- each builds its own `Client(host='localhost', …)`
+- they read `CLEANUP_*` / `SKIP_RATIO_CHECK` from `os.environ`
+- they call `date.today()`
+
+The test-design review found:
+- a spec-less `MagicMock` client verified by call arguments
+- a `disk_usage` side-effect list that hard-codes how many times production calls it (`tests/test_fetch_torrents.py:1959`)
+- an internal function patched out (`evaluate_old_release_demand`, `:1404`+)
+
+`check_old_releases_for_demand(today=…)` already shows the target pattern.
+
+## Acceptance criteria
+- `should_fetch_torrent(name, ratios, removed_history=None, *, skip_ratio_check=False)`. The env read moves to the caller, i.e. `load_config` from task 18 or `__main__`.
+- `cleanup_old_versions(client, *, today, skip_ratio_check, min_ratio)`
+- `cleanup_stagnant_torrents(client, *, today, window_days, min_ratio_delta)`
+- `enforce_disk_usage_limit(client_factory, threshold_percent, *, today, disk_usage=shutil.disk_usage, download_path=…)`, which still only connects once the threshold is exceeded.
+- Tests pass a fake client with real `get_torrents()`/`remove_torrent()` behavior: an in-memory list that records removals. Assert the resulting state rather than `assert_called_once_with`.
+- The `disk_usage` fake is driven by the fake client's removals (used space drops as torrents are removed), not by a fixed side-effect list.
+- The task-17 lock-in tests stay green.

tasks/20-transmission-double-fidelity.md

@@ -0,0 +1,24 @@
+# 20 — Transmission client double that can't drift
+
+**Priority**: P3 · **Class**: REFACTOR_REQUIRED · **Depends on**: 19
+
+## Problem
+Every test replaces `transmission_rpc` with a `types.ModuleType` stub whose `Client` is a bare
+`MagicMock`. Nothing checks that `Client(host=…, port=…, username=…, password=…)`, `get_torrents()`,
+`remove_torrent(id, delete_data=True)` or the torrent attributes the code reads (`name`, `id`,
+`ratio`, `added_date`, …) match the real library. The container installs Alpine's
+`py3-transmission-rpc`, whose version moves with `alpine:latest` + `apk upgrade`. `log_seed_ratios_via_http`
+also builds the raw RPC request by hand.
+
+CLAUDE.md says `transmission_rpc` is "apk-only, not pip-installable", but the cd-test-architecture
+review reports that the PyPI package `transmission-rpc` exists. **Check this first.**
+
+## Acceptance criteria
+- Verify: find which `transmission-rpc` version current Alpine ships (`apk info py3-transmission-rpc` in the image), and whether that version is on PyPI.
+- If it is:
+  - pin it in `requirements-test.txt`
+  - build the client double with `create_autospec(transmission_rpc.Client, instance=True)`, and torrents from the real `Torrent` shape or a validated fake
+  - correct the CLAUDE.md claim
+- If it isn't: introduce a thin `TransmissionAdapter` (`list_torrents() -> list[TorrentInfo]`, `remove(id)`) in production code. Then the only untested surface is the adapter itself, which task 23's scheduled job covers.
+- Either way, a test fails if production calls a `Client` method or keyword that the pinned library doesn't have.
+- Add a test for `log_seed_ratios_via_http` that asserts the request body matches the Transmission RPC spec fields it relies on.