Repository navigation
Conversation
Reviewers can attribute each backcheck mismatch to the enumerator,
backchecker or respondent from a pinned Review button on Comparison
Results Details; selecting several mismatch rows attributes them together.
A note is required for Backchecker and Respondent. Attributions are
appended to bc_attribution_{page_name_id} in the logs db with the user and
date, and lapse when either value changes. An Attribution log expander
lists the history.
The enumerator and backchecker tables gain adjusted error rates, the
variable table gains mismatch counts per source, and the summary shows the
share of mismatches attributed. A new optional error rate target
highlights each regular and adjusted rate column above it.
Backcheck mismatches can no longer be accepted: backchecks is dropped from
ACCEPT_CHECK_TYPES. The Backchecks page never changes survey or backcheck
data.
Closes #301
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Error Source now comes right after the Review button, and both stay pinned while scrolling. The Review column has a fixed width that fits its label instead of the default, which was too wide. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comparison Results Details now renders in an st.fragment, so selecting rows, changing its filters or opening the Review dialog reruns just that section instead of the whole report. Saving an attribution still reruns the whole app (st.rerun(scope="app")) so the rates are refreshed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Backchecks Summary gains an Error Rates section below Targets: one card for the total error rate and one for each category. Each card shows the enumerator adjusted error rate as a grey delta, and its help gives the backchecker adjusted rate and the counts behind the rate. A category with no values compared shows N/A. New compute_overall_error_rates returns the rates as OverallErrorRate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A category with no values compared had no delta line, so its card was shorter than the others. It now shows "No values compared" as a grey delta, and the cards use bordered columns, which stretch to the tallest card in the row. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Clearing a configured error-rate target does not persist across reruns, causing highlighting to return unexpectedly.
Review effort: Balanced
Findings: 1
What changed in this PR
Adds backcheck mismatch attribution, adjusted error rates, and related reporting while preventing backcheck acceptance.
Changes:
- Adds append-only mismatch attribution and review UI.
- Adds adjusted rates, targets, summary cards, and source counts.
- Updates tests and documentation.
| File | Description |
|---|---|
src/datasure/checks/backchecks/attribution.py |
Implements attribution logic and storage. |
src/datasure/checks/backchecks/compute.py |
Computes adjusted rates and source counts. |
src/datasure/checks/backchecks/models.py |
Adds the error-rate target setting. |
src/datasure/checks/backchecks/report_ui.py |
Adds review UI, metrics, and highlighting. |
src/datasure/checks/backchecks/settings_ui.py |
Adds error-rate target controls. |
src/datasure/processing/correction_log.py |
Disallows backcheck acceptance. |
tests/checks/backchecks/test_attribution.py |
Tests attribution rules and storage. |
tests/checks/backchecks/test_compute.py |
Tests rates, counts, and settings. |
tests/checks/backchecks/test_report_ui_attribution.py |
Tests attribution UI behavior. |
tests/checks/backchecks/test_settings_ui.py |
Tests target settings UI. |
tests/processing/test_corrections.py |
Tests rejection of backcheck acceptance. |
docs/USER_GUIDE.md |
Documents attribution and adjusted rates. |
docs/ARCHITECTURE.md |
Documents attribution-log storage. |
CHANGELOG.md |
Records user-facing changes. |
docs/changelog_guide.md |
Formats example code. |
.claude/subagents/py-format-lint/SUBAGENT.md |
Formatting-only cleanup. |
.claude/skills/surveycto-api/SKILL.md |
Formatting-only cleanup. |
.claude/skills/duckdb/SKILL.md |
Formatting-only cleanup. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Clearing the optional error rate target saved None, but loading the settings treated None as invalid and fell back to the configured target, so highlighting could come back after a rerun. A saved None now stays; only values outside 0-100 fall back. Also tidies two test assertions flagged by code scanning. Addresses review comments on #323. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Running pre-commit on all files reformatted code examples in three .claude skill files and docs/changelog_guide.md. Those changes are unrelated to #301, so they are restored to match the base branch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request Summary 🚀
Closes #301.
What does this PR do? 📝
Reviewers can now record who caused each backcheck mismatch, and the
Backchecks page reports an adjusted error rate next to the regular one. The
page never changes survey or backcheck data.
From the issue (#301):
backchecksis removed from the checktypes that can be accepted, so a backcheck mismatch can't be accepted as
valid. It can only be attributed.
dialog shows the survey and backcheck values read-only and offers
Enumerator, Backchecker, Respondent or Unattributed.
Review on one of the selected rows.
Unattributed.
bc_attribution_{page_name_id}, in thelogsdatabase, with the reviewer(from Corrections: record who made each correction log entry #321) and the date.
history.
there is no mismatch to attribute.
(values compared), and Unattributed mismatches always count.
each category and the total.
Unattributed mismatch counts.
Not in the issue — added during review with the requester:
"highlighted against the target", but no error-rate target existed; the only
target was backcheck coverage. A new optional setting under Tracking Options
sets one. Each regular and adjusted rate column above it is highlighted, in
both the enumerator and backchecker views. With no target set, nothing is
highlighted.
has one card for the total error rate and one for each category.
so all cards stay the same height.
columns and stay pinned while scrolling. The Review column has a fixed
width that fits its label.
own: selecting rows, changing its filters or opening Review reruns only that
section, not the whole report. Saving an attribution still reruns the whole
app so the rates refresh.
keys, so a Review click always resolves to the row that was clicked. This
changes the default row order users see.
Why is this change needed? 🤔
Backcheck results measure data collection quality, so the page must not offer
a way to make poor results disappear. Before this PR,
backcheckscould beaccepted through the correction log. Yet some mismatches are not the
enumerator's fault: the backchecker recorded a wrong value, or the respondent
changed their answer. Attribution records that without hiding anything:
Known gap, expected: duplicate and unmatched backcheck IDs are out of
scope (moved to #306 and #303; #302 is closed). Until #303 lands, the
existing Handle Duplicates setting keeps deciding which duplicates are
compared. #303 closes this gap.
How was this implemented? 🛠️
checks/backchecks/attribution.py. It holdsthe error sources, the note rule, building log entries (mismatch-only),
marking each comparison with its source (latest entry wins, lapse rule),
counts, both adjusted-rate formulas, the attributed share, and log storage.
It is tested without a running app, like
checks/outliers/review.py.check doesn't depend on column types. When the survey KEY is also the merge
ID, the survey and backcheck KEYs are the same column, and that case is
handled.
compute.py.formula.
report_ui.py.results, and every section reads from those marked results.
BackcheckSettingsgainserror_rate_target_percent(0–100).A saved value outside that range falls back to the default, as the coverage
target does.
How to test or reproduce ? 🧪
Automated.
just testgives 3634 passed, 6 skipped, andjust pre-commit-runpasses. The new tests cover:Manual, on a project with survey and backcheck data and at least one
configured backcheck column:
Unattributed, and the adjusted rates equal the regular rates.
disabled until you type a note. Save it. A toast appears, the row shows
Respondent, and the enumerator adjusted rate drops. The regular rate and
the mismatch counts don't change.
them all to Enumerator. All selected rows update.
the date, including earlier ones that were replaced.
come back. That mismatch is Unattributed again.
highlighted, and each rate column is checked against the target on its own.
whole page.
Screenshots (if applicable) 📷
Correction log shows who made the change. User is shown in the sidebar to the left.
Each error can be attributed. Clicking "Review" shows a dialog box.
Attribution log shows a history of changes including Error Source, User and Datetime
Backcheck Summary now includes overall Error Rates for backchecks with a delta showing adjusted rates.
Checklist ✅
widget rendered in Streamlit's test runner. The manual checks above
still need a browser run.)
explain why): 2054 lines changed in total, of which 922 are source, 1009
tests and 110 docs. The source change alone is under 1000. It runs over
because the issue's acceptance criteria call for tests of every rule,
and because of the extras listed above.
N/A, no new dependencies.
docs/USER_GUIDE.md(policy, both rates, attribution, new settings andcards),
docs/ARCHITECTURE.md(new table),CHANGELOG.md.branch,
feat/321-correction-log-user.Reviewer Emoji Legend
:code::smiley::+1::100:...and I want the author to know it! This is a way to highlight positive parts of a code review.
:star: :star: :star:And I am providing reasons why it needs to be addressed as well as suggested improvements.
:star: :star:And I am providing suggestions where it could be improved either in this PR or later.
:star:...and consider this a suggestion, not a requirement.
:question:This should be a fully formed question with sufficient information and context that requires a response.
:memo::pick:This does not require any changes and is often better left unsaid. This may include stylistic, formatting, or organization suggestions and should likely be prevented/enforced by linting if they really matter
:recycle:Should include enough context to be actionable and not be considered a nitpick.
🤖 Generated with Claude Code