Repository navigation
fix(workbook): refresh only rows created by its own sourcing - #92
Merged
Merged
Conversation
Refresh detected "newly sourced" rows by snapshotting row ids before sourcing and diffing afterward, so any row inserted concurrently by another writer (CSV import, API insert, another job) was treated as sourced and had every non-output runnable column run, including paid enrichments. materialize_source (ICP, crm_import and people_search paths) now returns the ids it inserted as an additive `added_row_ids` key, and refresh uses those instead of the snapshot diff, dropping both row-id queries. Follow-up to #74. Signed-off-by: debpalash <debash67@outlook.in> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #74.
Problem
refresh.pyfound "newly sourced" rows by snapshotting row ids before sourcing and diffing afterward. Rows inserted concurrently by another writer (CSV import, API insert, another job) were treated as sourced, so the scheduled refresh ran all their non-output runnable columns, including paid enrichments. The #74 ADR called this out as a limitation.Change
materialize_sourcesummaries now include an additiveadded_row_idskey, the ids that run inserted. It covers the ICP path (source_engine),crm_importandpeople_search. Existing keys are unchanged. Early-error returns leave the key out, so callers use.get("added_row_ids") or [].refresh.pycollectsadded_row_idsfrom each source column run and uses them in place of the snapshot diff. Both row-id snapshot queries are removed.docs/adr/20261012-refresh-dependent-and-new-row-cells.mdnow marks the limitation as resolved.Tests
test_concurrently_inserted_row_is_not_treated_as_sourced. The stubbed source inserts its own row (202) and a "concurrent" row (303), and reports only[202]. The test asserts that 202 gets every non-output runnable column, 303 and the existing row stay untouched, and nothing is dispatched. Fails on main (row 303 got formula/http/agent/research cells). Passes here.added_row_ids.added_row_idsand assert it equals the inserted rows ([]on a dedup re-run).uv run pytest -q: 2215 passed, 139 skipped, 0 failed.🤖 Generated with Claude Code