feat(corrections): add source and accept to the log, extract a shared correction form

[iabaako]

Pull Request Summary 🚀

What does this PR do? 📝

Adds the groundwork that the check-page correction work in #295 builds on. Closes #296 (stack position 1 of 11; #297 branches from here).

• source column in corr_log_{alias}, backfilled to corrections_page when an existing log is read. The empty-log schemas now include every column, status and status_reason among them.
• accept action. It records check_type, KEY, column (null for GPS), the value at the time of acceptance and a required reason. Replay and the generated replication script skip accept rows. correction_log.csv keeps them.
• Active-acceptance helper. An acceptance stays active only while the current value still matches the stored one. For GPS, latitude and longitude must both still match.
• Shared correction form, utils/correction_form.py. It takes a prefilled KEY, column and value, the allowed actions, a source and a widget-key namespace, and it can apply several entries as one all-or-nothing step. The Corrections page now uses it.
• Fix: CorrectionProcessor cache keys now include project_id.
• Fix: a new value of "0" now enables Apply.
• Correction Log shows the Source and check type. "Remove correction step" can remove accept rows.

 src/datasure/
 ├── processing/
+│   ├── correction_log.py    # log schema, source/accept constants, backfill
 │   └── corrections.py       # + accept_value, apply_corrections, active acceptances
 ├── replication/
 │   ├── package_builder.py   # skip accept rows in README counts
 │   └── script_generators.py # skip accept rows in generated script
 ├── utils/
+│   └── correction_form.py   # shared action / new value / reason form
 └── views/
     └── correction_view.py   # now renders the shared form

Why is this change needed? 🤔

Every check page in #295 (outliers, constraints, backchecks, duplicates, GPS) needs to write corrections and acceptances into the same log from its own tab. That needs a reusable form, a way to record where each entry came from, and an accept action that doesn't change the data. It also fixes two existing bugs:

• Two projects that share an alias were sharing cached corrected data, because _self is not hashed by st.cache_data.
• The Apply button used a truthiness check, so "0" counted as a missing value.

How was this implemented? 🛠️

• Log schema. The schema, the action names and the backfill now live in processing/correction_log.py. ensure_log_columns extends the old _ensure_status_columns backfill.
• Accept rows are written to the log like any other entry. _reapply_all_corrections skips them, and so do the replication script and the README summary.
• Cache keys. The four cached CorrectionProcessor methods use hash_funcs so that project_id is part of the cache key.
• Apply button. should_enable_apply_button now checks new_value is not None and new_value != "". An empty string is deliberately not a valid modify value; USER_GUIDE.md points users to "remove value" for that case.
• Atomic apply. apply_corrections saves the corrected data and then the log. If saving the log fails, it restores the corrected data.
• Corrections page. It passes its existing _handle_apply_correction as on_apply, so the page behaves as before.

Known follow-ups from the pre-PR review (not blocking this PR):

• The Corrections page doesn't use apply_corrections yet; its first real users are Corrections: resolve survey and backcheck ID problems from the duplicate cards #303 and Corrections: correct or accept GPS outliers from the GPS page #304.
• accept_value does not check that the KEY or column exists, but apply_corrections does.
• validate_correction_input compares KEY values without converting them to text, while the acceptance check converts first. That could matter for KEY columns that aren't text.
• The backfill happens when a log is read, not as a migration on disk.
• test_failed_log_save_leaves_corrected_data_unchanged fails if run together with only tests/views/test_correction_view.py, because the views conftest mocks Streamlit. It passes on its own and in the full suite.

How to test or reproduce ? 🧪

just test passes: 3333 passed, 6 skipped. just pre-commit-run passes. The new tests cover:

• Schema backfill: a legacy log without source loads with source = corrections_page and no data loss.
• Accept: corrected data is unchanged; replay skips the row and the generated script leaves it out; after the value changes, the acceptance shows as inactive (for GPS, when either latitude or longitude changes).
• Cache: two processors with the same alias and different project_ids return different corrected data. The tests use the real st.cache_data.
• Apply button: "0" enables Apply; None and "" do not.
• Shared form: the tests are in tests/views/test_correction_form.py; the existing tests/views/test_correction_view.py suite still passes.

Manual check:

1. Open a project with an existing correction log. The Correction Log shows a Source column set to corrections_page for old rows.
2. On the Corrections page, modify a value to 0. Apply is enabled and the change is saved.
3. Open two projects that share a form alias. Each one shows its own corrected data.

Screenshots (if applicable) 📷

Orange Box - Shows the source column with backfilled values for previous entries
Yellow Box - Shows successful modification of value to 0

Checklist ✅

• I have run and tested my changes locally
• I have limit this PR to less than 1000 lines of code change (if not, explain why): this PR has about 1,790 changed lines in src/ and about 1,170 lines of new tests. Corrections: add source and accept to the log, extract a shared correction form #296 is scoped as one foundation step: the schema, accept, form extraction and both fixes are all needed before Corrected data goes stale when prep steps change #297 can branch from it. More than half of the src/ diff comes from moving the form out of correction_view.py into correction_form.py.
• I have updated/added tests to cover my changes (if applicable)
• I have updated/added requirements to cover my changes (if applicable): N/A, no new dependencies
• I have run linting and formatting on any code changes (if applicable)
• I have updated the documentation (README, etc.) accordingly: CHANGELOG.md, docs/ARCHITECTURE.md, docs/USER_GUIDE.md
• I have reviewed and resolved any merge conflict

Reviewer Emoji Legend

	:code:	Meaning
😃👍💯	:smiley: :+1: :100:	I like this... ...and I want the author to know it! This is a way to highlight positive parts of a code review.
⭐⭐⭐	:star: :star: :star:	Important to fix before PR can be approved... And I am providing reasons why it needs to be addressed as well as suggested improvements.
⭐⭐	:star: :star:	Important to fix but non-blocking for PR approval... And I am providing suggestions where it could be improved either in this PR or later.
⭐	:star:	Give this some thought but non-blocking for PR approval... ...and consider this a suggestion, not a requirement.
❓	:question:	I have a question. This should be a fully formed question with sufficient information and context that requires a response.
📝	:memo:	This is an explanatory note, fun fact, or relevant commentary that does not require any action.
⛏	:pick:	This is a nitpick. This does not require any changes and is often better left unsaid. This may include stylistic, formatting, or organization suggestions and should likely be prevented/enforced by linting if they really matter
♻️	:recycle:	Suggestion for refactoring. Should include enough context to be actionable and not be considered a nitpick.

🤖 Generated with Claude Code

[Copilot]

Copilot review overview

🟡 Changes recommended

GPS mappings can omit coordinates, and unsupported form actions can incorrectly fall through to row deletion.

Review effort: Balanced
Findings: 1 · 1

Open (2)

• Reject unknown actions instead of deleting the selected row · New
• Reject GPS acceptance mappings without both coordinates · New

What changed in this PR

Adds correction provenance, acceptance tracking, atomic multi-entry application, and a reusable correction form.

Changes:

• Adds correction-log schema/backfill and active acceptances.
• Extracts shared correction UI and fixes cache scoping and zero-value submission.
• Excludes acceptances from replay and replication scripts while retaining audit records.

File	Description
src/​datasure/​processing/​correction_log.py	Defines log schema and vocabulary.
src/​datasure/​processing/​corrections.py	Adds acceptances, atomic apply, and cache fixes.
src/​datasure/​utils/​correction_form.py	Introduces the shared correction form.
src/​datasure/​views/​correction_view.py	Adopts the shared form and displays provenance.
src/​datasure/​replication/​package_builder.py	Preserves acceptances in audit exports.
src/​datasure/​replication/​script_generators.py	Omits acceptances from generated scripts.
tests/​processing/​test_corrections.py	Tests processor behavior and storage failures.
tests/​views/​test_correction_form.py	Tests shared form behavior.
tests/​views/​test_correction_view.py	Updates Correct Data view tests.
tests/​replication/​test_package_builder.py	Tests audit exports and counts.
tests/​replication/​test_script_generators.py	Tests acceptance omission.
docs/​ARCHITECTURE.md	Documents new modules.
docs/​USER_GUIDE.md	Documents acceptance and zero-value behavior.
CHANGELOG.md	Records the feature and fixes.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Copilot review overview

🔵 Needs a closer look

Acceptance submissions can record stale values and become inactive immediately.

Review effort: Balanced
Findings: None

Resolved since last review (2)

• Reject unknown actions instead of deleting the selected row
• Reject GPS acceptance mappings without both coordinates

Previously missed (1)

In code that hasn't changed since last review

Validate accepted value matches current row before logging

src/​datasure/​processing/​corrections.py:658

The acceptance path only verifies that the KEY exists; it never compares entry.current_value with the current row. A stale form can therefore log an acceptance that is immediately inactive (and the log does not contain the value at the time of acceptance). Before returning, derive the snapshot from data or reject a mismatch for the selected column—and for both GPS coordinates.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
96.3% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud