Skip to content

fix(stac): declare the label property dataset items actually carry - #64

Open
cdarnell wants to merge 5 commits into
hotosm:developfrom
cdarnell:fix/480-label-property-matches-stamped-labels
Open

fix(stac): declare the label property dataset items actually carry#64
cdarnell wants to merge 5 commits into
hotosm:developfrom
cdarnell:fix/480-label-property-matches-stamped-labels

Conversation

@cdarnell

@cdarnell cdarnell commented Aug 21, 2026

Copy link
Copy Markdown

Closes part of hotosm/fAIr#480. Follows on from the review there; @kshitijrajsharma asked for minimal viable changes with proven references, so this is only the label-encoding half.

The problem

fair/datasets.py and fair/stac/builders.py disagree about where the class lives. The materializer stamps it onto properties.label:

# fair/datasets.py:70
feature.setdefault("properties", {})[LABEL_CLASS_PROPERTY] = index   # was: ["label"]

The builder declared something else:

# fair/stac/builders.py, before
label_properties if label_properties is not None else (None if label_type == "raster" else ["class"])

and republished label_classes — which is the OSM filter spec, per _osm_filters' own docstring — verbatim as label:classes.

So an item built by this library said the class lives in class, and in building/highway, with values "yes"/"house"/"*". The file it describes has label with integer values. No overlap.

Why that's wrong

The Label Extension v1.0.1, which every dataset item declares via DATASET_EXTENSIONS, defines:

  • label:properties — "the names of the property field(s) in each Feature of the label asset's FeatureCollection that contains the classes"
  • Class Object name — "The property key within the asset's each Feature corresponding to class labels"

Both must name a property that exists on the features.

Two things worth stating plainly so this isn't oversold:

  • It validates today. validate_item() and pystac are both green — the schema only checks types and required-ness. This is semantic non-conformance, not a schema failure.
  • Nothing inside fAIr reads these fields. The pipelines burn binary masks. The effect is limited to external consumers, which is what #480 and #481 are about.

The change, commit by commit

Each one is green on its own, so it can be reviewed (or dropped) in stages.

Commit What Why
refactor(stac): name the label class property in one place LABEL_CLASS_PROPERTY = "label" in constants.py; _stamp_class_label uses it A literal in two modules is how they drifted. No behaviour change — it just gives the next commits something to point at.
fix(stac): declare the label property that dataset items actually carry default label:properties becomes [LABEL_CLASS_PROPERTY] ["class"] names a key on no feature this library writes. One token, and the half that matters most.
fix(stac): publish class objects that name the stamped property label:classes built from what _stamp_class_label assigns; tag mapping moved to label:description A Class Object's name must also be a real property key. Without this the item is still non-conformant, just differently.
test(stac): pin dataset label semantics to what the materializer writes TestDatasetLabelSemantics, 6 tests The bug was two modules disagreeing, so the test round-trips between them and fails if either drifts.
docs(schemas): document how dataset labels are encoded new subsection in docs/schemas.md The Dataset section never said what these fields should contain, which is how the drift went unnoticed.

The change

fair/stac/constants.py LABEL_CLASS_PROPERTY = "label" — one constant, so writer and declaration can't drift again
fair/datasets.py stamp through the constant (no behaviour change)
fair/stac/builders.py default label:properties to it; publish Class Objects naming it; preserve the OSM tag mapping in label:description

Output for a two-class request:

"label:properties": ["label"],
"label:classes": [{"name": "label", "classes": [1, 2]}],
"label:description": "\n\nClass values in `label` (0 = background): 1 = building=yes|house; 2 = highway=*."

against a feature the materializer produced:

{"osm_id": 1, "osm_type": "way", "tags": {"building": "house"}, "label": 1}

No dataset needs rebuilding — this changes what is declared, not the bytes in S3.

Backward compatibility

A caller passing label_properties is describing its own label file and is left completely alone: label_classes and label_description publish unchanged. label:properties stays null for label:type: "raster". Only the default path changes, and its old value (["class"]) matched nothing this library writes.

Tests

TestDatasetLabelSemantics in tests/test_builders.py. The main one round-trips against _stamp_class_label itself, so it fails if either side drifts:

AssertionError: assert 'class' in {'label': 1, 'osm_id': 1, 'osm_type': 'way', 'tags': {...}}

That's what the suite reports with the fix reverted. Also covered: the ["*"] sentinel no longer reaching the catalog, the tag mapping surviving into label:description, explicit label_properties passing through untouched, and raster still declaring null.

just test 413 passed (was 407), coverage 96.10%, just lint clean, scripts/validate_stac_items.py OK on all 8 shipped items.

Your existing suite caught a bug in my first draft — test_dataset_param_and_finetune_validation_error_paths passes a class entry with no classes key, which my description helper assumed. Hardened.

Not in this PR

Deliberately left out to keep it minimal; happy to follow up on any, separately:

  • Per-item license (dataset items carry none; the labels are OSM-derived, which matters for #481)
  • fair:source_imagery never set — the fAIr backend passes source_imagery_href= where the property is built from source_imagery=
  • item_assets and real extents on the three collections
  • The four data/sample/* items, which use a third convention again and are what implementers copy — worth aligning once you've confirmed this one

One open question

I went with ["label"] because it describes the files that already exist. The shipped sample uses label:properties: ["building"] with tag values, which is more self-describing for a consumer.

Worth knowing what that costs before you pick: the OSM tag currently sits nested under properties.tags, so ["building"] only becomes a true statement once every existing labels.geojson is rewritten to lift the matched tag to a top-level property, and every dataset item is re-published to match. There is also the ordering question — the integer index carries class order, which multi-class training needs, and tag values have no inherent order.

Since label:properties is a list, there is an additive middle path: keep label for the class index and add the tag alongside it, declaring both with a Class Object each. That still needs the file rewrite, but nothing already reading label breaks.

Happy to do any of the three — just say which, and I will keep it to the same minimal shape.

cdarnell and others added 5 commits August 21, 2026 09:20
`_stamp_class_label` writes the class index onto a hardcoded `"label"` key.
The STAC builder has to declare that same key, so a literal in two modules is
a drift risk. No behaviour change; this only gives the next commits something
to point at.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Label extension defines `label:properties` as "the names of the property
field(s) in each `Feature` of the label asset's `FeatureCollection` that
contains the classes". Vector items defaulted it to `["class"]`, a key that
appears on no feature `fair.datasets` writes — it stamps `properties.label`.

A consumer following the item to read the class found nothing. Items still
validate, since the schema only checks types and required-ness, and nothing
inside fAIr reads the field, so this only ever affected external consumers.

Callers that pass `label_properties` explicitly are untouched, and raster
labels still declare null. No dataset needs rebuilding.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`label_classes` is the OSM *filter* spec — tag key to accepted tag values, as
`_osm_filters` uses it — and it was republished verbatim as `label:classes`.
But the extension defines a Class Object's `name` as "the property key within
the asset's each `Feature` corresponding to class labels", and its `classes`
as the values that key takes. The published objects named `building` and
`highway` with values like `"yes"` and the internal `["*"]` wildcard, while
the features carry `label` with integer indices.

Build the Class Objects from what `_stamp_class_label` assigns instead. The
tag mapping that is lost in that translation moves into `label:description`,
the only field able to express it, so no information is dropped.

Applies only when the caller has not declared its own `label_properties`; the
mapping helper tolerates class entries with no `classes` key, which
`test_dataset_param_and_finetune_validation_error_paths` passes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The defect was two modules disagreeing, so the main test round-trips between
them: stamp features with `_stamp_class_label`, build an item, then assert
every key in `label:properties` and every `label:classes[].name` is present
on those features with a matching value. It fails if either side drifts.

Reverting the fix turns it into the defect verbatim:

    AssertionError: assert 'class' in {'label': 1, 'osm_id': 1, ...}

Also covers the `["*"]` sentinel staying out of the catalog, the tag mapping
surviving into `label:description`, explicit `label_properties` publishing
untouched, and raster still declaring null.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Dataset section listed required properties but never said what
`label:properties` or `label:classes` should contain, which is how the two
modules drifted. Records the convention with the extension's own wording, the
resulting JSON, and the rule for callers supplying their own label files.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@cdarnell

Copy link
Copy Markdown
Author

@kshitijrajsharma this is the minimal-viable slice from hotosm/fAIr#480 — label encoding only, split into five commits so it can be reviewed or dropped in stages. The other items from the review are listed at the bottom as follow-ups rather than folded in.

One open question at the end: I went with label:properties: ["label"] because it describes the files that already exist, but the shipped sample uses ["building"]. Happy to switch if you'd rather have that shape long-term.

@codecov

codecov Bot commented Aug 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.09%. Comparing base (9a1c6a4) to head (56c04b9).

Additional details and impacted files
@@             Coverage Diff             @@
##           develop      #64      +/-   ##
===========================================
+ Coverage    96.07%   96.09%   +0.01%     
===========================================
  Files           33       33              
  Lines         2907     2921      +14     
===========================================
+ Hits          2793     2807      +14     
  Misses         114      114              
Flag Coverage Δ
fair 96.09% <100.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant