[rollroyces]

What changes from the review

Rewritten to implement 0072 §Custom, which replaces what #1246 did. The maintainer's point was correct: take(MAX_NEW_PER_SYNC) followed by a failed run replays the same 200 items forever and never reaches item 201 — exactly what 0072 lists under Rejected.

crates/utopia-server/src/ingest_sources.rs:

• SyncStats.incomplete: bool (was truncated) — set only when processing stops with items still behind.
• stats.discovered = items.len() — the response size, not the number imported. The previous run logged (0 > 200) because discovered was never set.
• sync_custom walks the whole response. A local changed counter is charged only by IngestAction::Created / Updated; Unchanged / Moved / Duplicate / Tombstoned cost nothing. The loop breaks when changed reaches MAX_NEW_PER_SYNC, setting incomplete.
• sync_source gained a third match arm, Ok(stats) if stats.incomplete, which writes the run as failed. Since last_successful_sync_start only reads status = 'ok', the checkpoint stays at the last complete round and the next run re-sends the same since. The run still records its created / updated counts, per 0072's "Keep the committed counts visible for incomplete runs".

No store change: last_successful_sync_start already filtered on status = 'ok', so writing failed is all that is needed to hold the cursor.

Test

a_truncated_custom_batch_resumes_on_the_next_run in source_checkpoint_tests.rs — serves 201 fresh items and calls sync_source twice:

1. First run imports 200, ends failed, error mentions stopping with more behind.
2. Second run re-receives the same 201; the first 200 are now Unchanged and cost no budget, so item 201 lands. The source ends ok.

Two CI issues from the previous review are fixed:

• cargo fmt — ran it; the diff is clean.
• source_sync_runs_source_id_fkey — the sources INSERT now runs before start_run, so the foreign key resolves. (The old test inserted the source after creating a run row.)

Scope

Refs #1212, not Closes — this covers the Custom half only. The GitHub and Jira halves of #1212 need the timestamp-group completion that 0072 specifies and are not in this PR.

Verification

CI is green on this branch — build, lint, test, migrations, backend, web all pass. The test job runs against a real Postgres, so a_truncated_custom_batch_resumes_on_the_next_run actually executed there rather than skipping.

Locally: cargo fmt --all --check and cargo clippy -p utopia-server --all-targets -- -D warnings both pass. Clippy caught two problems in the first draft of this change, both fixed here — a field_reassign_with_default on the discovered assignment and an E0382 borrow in the new test.

[WaylandYang]

Thanks, this is the shape 0072 asks for: the whole response is walked, only changed items count, and the second run reaches item 201. Before merge: (1) each capped run currently raises a SOURCE_SYNC_FAILED error alert via finish_sync(Some(err)); please record the run as failed with the message but skip the alert (see #1212). (2) Update 0072's status line and replace the #1246 reference in §Custom with this PR. (3) Add the 450-items/three-runs case from 0072's acceptance list, and assert discovered == 201. (4) Say "changed 200 items" rather than "read" in the run error.

[WaylandYang]

Heads-up: #1359 (the GitHub/Jira half of 0072) changes the same sync_source match and SyncStats, so please rebase onto it once it lands and keep its GitHub/Jira branch. 0072 also asks for a one-time checkpoint reset for custom sources (they may already have skipped items); please include that here alongside the points from my last review.