feat(outliers): correct or accept outliers and constraint violations from the check page

CHANGELOG.md

@@ -33,6 +33,55 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   `apply_correction_entries`) renders the action, new-value and reason inputs
   for a prefilled KEY/column/current value, with namespaced widget keys. The
   Correct Data page now uses it — #296
+- **Outliers and constraints corrections**: Each row of the constraint
+  violations and outlier inspection tables has a Review button (a pinned
+  `st.column_config.ButtonColumn`) that opens the shared correction form in a
+  dialog, prefilled with the row's KEY, column and current value, to modify the
+  value, remove it or accept it as valid (source and check type
+  `outliers`/`constraints`). Accepted flags are hidden and left out of the
+  metrics unless "Show reviewed" is on (then they are highlighted green), and
+  come back if the value changes or
+  the acceptance is removed. Accepting a hard violation needs a confirmation.
+  A "Show only flagged values" toggle (on by default) sits above both tables;
+  the outlier inspection table previously always listed unflagged values too.
+  With "Show reviewed" on, values whose current value comes from a modify or
+  remove correction are also highlighted green, with a "Corrected" badge and
+  the correction reason (new `CorrectionProcessor.get_active_corrections`);
+  unlike accepted flags, corrected values that are still flagged stay visible
+  and counted. A "Show only reviewed" toggle lists only accepted and
+  corrected rows and disables the other two toggles while on (toggle values
+  are a `review.TableFilters`, applied by `review.filter_table`). The outlier
+  inspection table now hides its index. Styled tables
+  are built with new `ui_utils.row_styler`, which keeps values displayed as in
+  the unstyled table (pandas' default Styler formatting showed `150` as
+  `150.000000` and missing values as `nan`).
+  Flag review logic lives in the new Streamlit-free
+  `src/datasure/checks/outliers/review.py`; `outliers_report` takes the
+  dataset `alias`. Removed the unused `_render_outlier_table`.
+  `queue_notice` gains a `toast` level, and `show_queued_notices` returns
+  whether it showed anything. New `ui_utils.ensure_styler_limit` raises
+  pandas' process-wide `styler.render.max_elements` under a lock and never
+  lowers it, so concurrent sessions can't cut it below what another render
+  needs; it replaces the `pd.set_option` calls in the summary, missing and
+  progress checks, which lowered the limit to fit their own tables and could
+  crash other styled tables. New `ui_utils.styled_dataframe` renders a Styler
+  after raising the limit to fit it; the results tables and the Correction
+  Log use it. A soft-violation acceptance no longer hides a value that has
+  since become a hard violation (e.g. after bounds are tightened): it needs a
+  new hard acceptance. A correction that failed to reapply to new prep data
+  is never shown as Corrected, even if the data holds its new value. Review
+  on a KEY whose rows hold different values of the column shows a warning
+  instead of the form, since a correction changes every row with the KEY
+  (`review.key_has_conflicting_values`). Survey fields added through "Show
+  more columns" that share a name with a results or review column get a
+  " (survey)" suffix, even while the review columns are hidden. A Survey KEY
+  named "review status" or "review reason" turns review off with a warning
+  instead of being overwritten — #298
+- **Correction log severity**: New `severity` column, `hard` on acceptances of
+  hard constraint violations (null otherwise and for legacy logs).
+  `CorrectionEntry.severity` sets it and is rejected on non-accept actions,
+  on acceptances of other checks, and with any value other than `hard`.
+  Hard acceptances are highlighted in the Correction Log — #298
 
 ### Fixed
 
@@ -43,6 +92,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   string still does; use "remove value" to blank a cell — #296
 - **Correction log schema**: Removing the last correction entry now leaves an
   empty log with the full schema, including status columns — #296
+- **Constraint violations**: A value past a hard bound was reported as a soft
+  violation whenever a soft bound on the same side was set (for example,
+  above the hard maximum read "above soft maximum"), so hard violations were
+  undercounted. Hard bounds are now tested first
+  (`compute_constraint_violations`) — #298
 
 ## [1.1.0] - 2026-09-21
 

CONTRIBUTING.md

@@ -150,9 +150,13 @@ Every Streamlit view must render its chrome through the shared helpers in
   (delete/remove/restart) — do not invent per-view confirm flows with
   session-state flags, expanders, or inline warnings.
 - `queue_notice(scope, level, message)` for any success/warning/error message
-  raised just before an `st.rerun()` (including a `confirm_dialog` callback,
-  which reruns), with `show_queued_notices(scope)` where it should appear on
-  the next run. Rendering it directly gets cleared by the rerun.
+  or toast raised just before an `st.rerun()` (including a `confirm_dialog`
+  callback, which reruns), with `show_queued_notices(scope)` where it should
+  appear on the next run. Rendering it directly gets cleared by the rerun.
+- Render a pandas Styler with `styled_dataframe`, or call
+  `ensure_styler_limit(cells)` before rendering it. Never set
+  `styler.render.max_elements` with `pd.set_option`: the option is shared by
+  every session, and lowering it can break another session's styled table.
 - Use `st.divider()` for horizontal rules, never `st.write("---")`.
 - Icons are Material shortcodes (`:material/check_circle:`), not emoji
   shortcodes (`:white_check_mark:`).

docs/USER_GUIDE.md

@@ -469,6 +469,8 @@ Check type column shows which check it applies to. It stays in effect only
 while the value is unchanged. Accept entries are kept in `correction_log.csv`
 in the replication package but are not part of the corrections script. You can
 remove an accept entry with "Remove correction step" like any other entry.
+Accepting a hard constraint violation sets Severity to `hard`, and those rows
+are highlighted in red.
 
 To blank a cell, use "remove value": "modify value" needs a non-empty new
 value (`0` is valid).
@@ -827,6 +829,45 @@ Visual analysis:
 - **Box Plot**: Distribution with outliers highlighted
 - **Table**: All records with outlier indicators
 
+##### Correcting or Accepting Flagged Values
+
+Above each of the constraint violations and outlier inspection tables,
+**Show only flagged values** (on by default) limits the table to flagged
+values. Turn it off to see every checked value.
+
+Each row of the constraint violations table and the outlier inspection table
+starts with a **Review** button. Click it to open a correction form in a
+dialog, with the KEY, column and current value filled in. Choose an action,
+enter a reason and click "Apply":
+
+- **modify value** or **remove value** corrects the data. The page reloads, and
+  the flag is updated or disappears.
+- **accept** records that the flagged value is correct. The flag is hidden
+  and no longer counted in the metrics. Turn on "Show reviewed" to see
+  accepted flags highlighted in green, with a Reviewed badge and the reason.
+
+"Show reviewed" also highlights corrected values in green, with a Corrected
+badge and the correction's reason. A corrected value that is now in range is
+no longer flagged, so turn off "Show only flagged values" to see it. A
+corrected value that is still flagged stays in the table and in the metrics
+until it is fixed or accepted. The badge clears if the value changes again.
+
+Turn on **Show only reviewed** to list only accepted and corrected values. While
+it is on, the other two toggles are disabled.
+
+Outlier and constraint acceptances are separate: accepting an outlier does not
+accept a constraint violation on the same value. Accepting a **hard**
+constraint violation needs an extra confirmation. An accepted flag comes back
+if the value changes, or if you remove the acceptance on the Correct Data page.
+Every entry appears in the Correction Log with source `outliers` or
+`constraints`.
+
+Corrections and acceptances apply to every row with the same KEY. If a KEY is
+on more than one row with different values in the flagged column, Review shows
+a warning instead of the form, because a correction would change all those
+rows. Give each record a unique KEY in the source data first. The Duplicates
+check lists duplicated KEYs.
+
 ---
 
 ### 6. Enumerator Stats Report

src/datasure/checks/missing.py

@@ -29,6 +29,7 @@
     save_check_settings,
     trigger_save,
 )
+from datasure.utils.ui_utils import ensure_styler_limit
 
 TAB_NAME = "missing"
 
@@ -930,7 +931,7 @@ def missing_columns(
     if not mv_data_filtered.empty:
         cmap = sns.light_palette("pink", as_cmap=True)
         styler_limit = mv_data_filtered.shape[0] * mv_data_filtered.shape[1]
-        pd.set_option("styler.render.max_elements", styler_limit)
+        ensure_styler_limit(styler_limit)
 
         st.dataframe(
             mv_data_filtered.style.format(
@@ -1114,7 +1115,7 @@ def missing_compare(
     else:
         cmap = sns.light_palette("pink", as_cmap=True)
         styler_limit = group_by_data.shape[0] * group_by_data.shape[1]
-        pd.set_option("styler.render.max_elements", styler_limit)
+        ensure_styler_limit(styler_limit)
 
         st.dataframe(
             group_by_data.style.format(subset=compare_col, precision=2)

src/datasure/checks/outliers/compute.py

@@ -920,15 +920,17 @@ def compute_constraint_violations(
         for col in outlier_cols:
             col_df = data.select([survey_key, col])
 
+            # Hard bounds are tested before soft ones: a value past a hard
+            # bound is also past the soft bound inside it.
             violation_expr = (
                 pl.when((hard_min is not None) & (pl.col(col) < hard_min))
                 .then(pl.lit(f"Value is below hard minimum {hard_min}"))
+                .when((hard_max is not None) & (pl.col(col) > hard_max))
+                .then(pl.lit(f"Value is above hard maximum {hard_max}"))
                 .when((soft_min is not None) & (pl.col(col) < soft_min))
                 .then(pl.lit(f"Value is below soft minimum {soft_min}"))
                 .when((soft_max is not None) & (pl.col(col) > soft_max))
                 .then(pl.lit(f"Value is above soft maximum {soft_max}"))
-                .when((hard_max is not None) & (pl.col(col) > hard_max))
-                .then(pl.lit(f"Value is above hard maximum {hard_max}"))
             )
 
             col_df = safe_to_numeric(col_df, col)