Skip to content

Empty prep or corrected tables fall back to raw data #315

Description

@iabaako

Description

When a prep step removes every row, check pages show the original raw data. The empty prep result is what they should show.

load_data_with_fallback
(output_view_template.py:96-131)
tries corrected, then prep, then raw, and returns the first table that is not empty. It cannot tell a table that doesn't exist from one that exists with zero rows. A valid zero-row result in corrected or prep therefore falls through to raw.

The Prep page has the same blind spot. A dataset whose prep data has zero rows takes the "No data available to prepare" path in
prep_view.py.
That path hides the Add and Remove step controls, so the user cannot remove the step that emptied the dataset.

This bug predates #297. Before #297 the corrected table stayed stale and non-empty. #297 rebuilds it correctly as an empty table, which then falls through to raw.

Raised by Copilot in review of #314.

Steps to reproduce

  1. Import a dataset and open Prep.
  2. Add a "remove row(s)" step whose condition matches every row.
  3. Open a check page. It shows all the raw rows.
  4. Go back to Prep. The tab says "No data available to prepare" and has no way to remove the step.

Expected

  • Check pages fall back to the next table only when a table does not exist, not when it has zero rows.
  • Check pages handle a zero-row dataset without crashing, and say there is no data.
  • The Prep page still shows the Change Log and the Remove step control for a dataset whose prep data has zero rows.

Acceptance criteria

  • load_data_with_fallback checks whether each table exists (for example with duckdb_table_exists) rather than whether it has rows.
  • Every check module is audited for zero-row input, with tests where it previously relied on non-empty data.
  • The Prep page lets the user remove steps when prep data has zero rows.
  • Tests cover a zero-row prep table and a zero-row corrected table.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions