Add MUP-based coverage-gap diversity metric

[sedirmohammed]

Summary

• add a Diversity data-quality dimension and a table-level diversity_coverageGap metric
• calculate the exact MUP-induced uncovered pattern space using finite-domain DNF counting
• parse the final field of every MUP row as its actual coverage and validate it against mincov
• add a dedicated Streamlit MUP uploader with positional dataset-attribute mapping and threshold inference
• document the Python and GUI workflows

Motivation

METIS did not yet expose coverage-based diversity assessment from a discovered Maximal Uncovered Pattern frontier. This adds the paper-defined coverage-gap calculation while preserving METIS's existing workflow and higher-is-better score convention. The exact coverage gap remains available in the result explanation, while DQvalue stores its complement, the coverage-space score.

User impact

Users can upload a dataset and its corresponding MUP file in the Streamlit GUI, select the discovery attributes, and calculate the exact coverage gap. Python users can configure the same metric with a MUP file path.

For the supplied Blue Nile case at mincov=19000, the implementation reproduces:

• 380,122 uncovered patterns
• 380,160 total patterns
• coverage gap: 0.9999000420875421
• coverage-space score: 0.0000999579124579

The 40 trailing MUP coverage values are parsed and validated; they range from 75 to 17,488.

Validation

• Blue Nile reference calculation reproduced exactly after removing the standalone test folder
• .venv/bin/python -m compileall -q metis gui
• git diff --check
• Streamlit health endpoint verified locally

[Copilot]

Pull request overview

This PR introduces a new Diversity data-quality dimension and a table-level diversity_coverageGap metric that computes the exact MUP-induced uncovered pattern space via finite-domain DNF union counting, with supporting GUI workflow and unit tests.

Changes:

• Add DQDimension.DIVERSITY and register the new diversity_coverageGap metric in the metric registry.
• Implement exact uncovered-pattern counting (CoverageSpaceCounter) and MUP parsing/validation (including trailing per-MUP coverage validation vs mincov).
• Add a dedicated Streamlit MUP uploader/editor and document the new Python + GUI workflows, plus focused unit tests.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/test_diversity_coverage_gap.py	Adds unit tests for exact counting and the new diversity metric behavior.
README.md	Documents the new diversity_coverageGap metric and adds it to the metric list.
metis/utils/dq_dimension.py	Adds a new DIVERSITY DQ dimension.
metis/metric/diversity/diversity_coverageGap.py	Implements the metric assessment, MUP reading/parsing, and result explanation fields.
metis/metric/diversity/diversity_coverageGap_config.py	Adds validated config model for MUP source and positional attribute mapping.
metis/metric/diversity/coverage_space_counter.py	Implements exact finite-domain DNF union counting with canonicalization.
metis/metric/diversity/__init__.py	Exposes the diversity metric package API.
metis/metric/__init__.py	Registers/imports the new diversity metric so it appears in the metric registry.
gui/ui/pages/metrics_page.py	Uses the new dedicated MUP editor for diversity_coverageGap configuration.
gui/ui/icons.py	Adds an icon mapping for the new “Diversity” dimension.
gui/ui/components/config_editors/mups_editor.py	Implements Streamlit uploader + positional attribute mapping + mincov inference.
docs/GUI.md	Documents the dedicated GUI editor behavior for the new metric.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.