diff --git a/rfcs/0048-geometry-geography.md b/rfcs/0048-geometry-geography.md index 8fe8aa5..c19f066 100644 --- a/rfcs/0048-geometry-geography.md +++ b/rfcs/0048-geometry-geography.md @@ -1,6 +1,6 @@ # Geometry and Geography Data Types -Champion: [Sander Bylemans](https://github.com/SBylemans) +Champion: TBD Authors: * [Sander Bylemans](https://github.com/SBylemans) @@ -19,13 +19,13 @@ Applies to: ## Summary -This RFC introduces two new values for `logicalType` in ODCS — `geometry` and `geography` — together with a dedicated set of `logicalTypeOptions` (`subType`, `crs`, `dimensions`, `algorithm`, and `encoding`). `geometry` represents shapes in a flat-earth (planar/Euclidean) coordinate system; `geography` represents coordinates on a round-earth (spherical/ellipsoidal) model. Both align with ISO 19125-1 (Simple Features for SQL), Apache Iceberg v3, GeoArrow, and GeoParquet. The physical encoding format (WKT, WKB, GeoJSON, etc.) is captured by the `encoding` option in `logicalTypeOptions`, while `physicalType` carries the target system's native column type. +This RFC introduces two new values for `logicalType` in ODCS — `geometry` and `geography` — together with a dedicated set of `logicalTypeOptions` (`subType`, `crs`, `dimensions`, `algorithm`, `encoding`, `bbox`, `orientation`, and `epoch`). `geometry` represents shapes in a flat-earth (planar/Euclidean) coordinate system; `geography` represents coordinates on a round-earth (spherical/ellipsoidal) model. Both align with ISO 19125-1 (Simple Features for SQL), Apache Iceberg v3, GeoArrow, GeoParquet 2.0, and the Apache Parquet native `GEOMETRY` / `GEOGRAPHY` logical types. The physical encoding format (WKT, WKB, GeoJSON, etc.) is captured by the `encoding` option in `logicalTypeOptions`, while `physicalType` carries the target system's native column type. ## Motivation ### Why are we doing this? -Geospatial data is a first-class data shape in modern analytics, logistics, real estate, infrastructure, and scientific datasets. Virtually every major database and data lakehouse ships a native geometric or geographic type — PostGIS, BigQuery `GEOGRAPHY`, Snowflake `GEOGRAPHY`, Databricks (Delta Lake + Iceberg v3), DuckDB, Apache Sedona, Hive, Presto/Trino, and others. Formats like GeoParquet and GeoArrow have standardized the columnar representation of geospatial data. +Geospatial data is a first-class data shape in modern analytics, logistics, real estate, infrastructure, and scientific datasets. Virtually every major database and data lakehouse ships a native geometric or geographic type — PostGIS, BigQuery `GEOGRAPHY`, Snowflake `GEOGRAPHY`, Databricks (Delta Lake + Iceberg v3), DuckDB, Apache Sedona, Hive, Presto/Trino, and others. Formats like GeoParquet 2.0 and GeoArrow have standardized the columnar representation of geospatial data, and Apache Parquet now ships native `GEOMETRY` and `GEOGRAPHY` logical types. ODCS today has no standard way to describe a geospatial column. Authors are forced to use `logicalType: string` (for WKT) or leave the column type opaque, which: @@ -37,7 +37,7 @@ Adding `geometry` and `geography` as first-class logical types matches the prece ### Use cases -1. **Parcel and land-registry data**: A data contract declares a `parcel_boundary` column as `logicalType: geometry` with `subType: Polygon` and `crs: urn:ogc:def:crs:EPSG::28992`, letting GIS tools load the correct projection without manual configuration. +1. **Parcel and land-registry data**: A data contract declares a `parcel_boundary` column as `logicalType: geometry` with `subType: Polygon` and `crs: EPSG:28992`, letting GIS tools load the correct projection without manual configuration. 2. **Ride-sharing and logistics**: A `pickup_location` column is declared as `logicalType: geography` (round-earth), ensuring that distance calculations account for Earth's curvature. 3. **Sensor and IoT data**: A `gps_track` column is typed `logicalType: geography`, `subType: LineString`, `dimensions: 3` (XYZ with altitude), enabling spatial analytics over device trajectories. 4. **Data lakehouse migration**: Teams migrating geospatial tables from PostGIS to Snowflake or from Hive to Iceberg v3 use the contract to generate correct DDL without hand-editing. @@ -45,7 +45,7 @@ Adding `geometry` and `geography` as first-class logical types matches the prece ### Alignment with guiding values -- **Small standard over large**: This RFC adds two `logicalType` values and five optional `logicalTypeOptions`. No new top-level structures. +- **Small standard over large**: This RFC adds two `logicalType` values and six `logicalTypeOptions`. No new top-level structures. - **Interoperability over readability**: The shape maps onto PostGIS, BigQuery, Snowflake, Databricks, DuckDB, Apache Sedona, GeoParquet, GeoArrow, and Apache Iceberg v3. - **Non-breaking**: `geometry` and `geography` are new optional `logicalType` values. Existing contracts are unaffected. @@ -60,13 +60,13 @@ Geospatial data comes in two flavours: The physical encoding (how the bytes are laid out on disk) is separate from both the logical type and the target system's column type. It is captured by the `encoding` option in `logicalTypeOptions`: -| `encoding` value | Description | -| ---------------- | -------------------------------------------------------------------- | -| `wkt` | Well-Known Text — human-readable string, e.g. `POINT (4.9 52.4)` | -| `wkb` | Well-Known Binary — compact binary encoding defined by OGC | -| `geojson` | GeoJSON encoding (JSON object with `type` and `coordinates`) | -| `ewkt` | Extended WKT — PostGIS extension that embeds the SRID in the string | -| `ewkb` | Extended WKB — PostGIS extension that embeds the SRID in binary form | +| `encoding` value | Description | +| ---------------- | -------------------------------------------------------------------------------------------------------------- | +| `wkb` | Well-Known Binary — compact binary encoding defined by OGC. **Canonical** for GeoParquet 2.0 and Parquet native `GEOMETRY` / `GEOGRAPHY` (the only encoding they accept). | +| `wkt` | Well-Known Text — human-readable string, e.g. `POINT (4.9 52.4)` | +| `geojson` | GeoJSON encoding (JSON object with `type` and `coordinates`) | +| `ewkt` | Extended WKT — PostGIS extension that embeds the SRID in the string | +| `ewkb` | Extended WKB — PostGIS extension that embeds the SRID in binary form | `physicalType` carries the target system's native column type (e.g. `GEOMETRY` in PostGIS, `GEOGRAPHY` in BigQuery, `STRING` in Databricks). See the [Physical type mapping](#physical-type-mapping) section below. @@ -82,10 +82,13 @@ The physical encoding (how the bytes are laid out on disk) is separate from both | Option | Applies to | Required | Type | Description | | ------------ | --------------------- | -------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `subType` | geometry, geography | No | string | The geometry subtype per ISO 19125-1. One of `Point`, `LineString`, `Polygon`, `MultiPoint`, `MultiLineString`, `MultiPolygon`, `GeometryCollection`. When omitted, any subtype is accepted. | -| `crs` | geometry, geography | No | string | The Coordinate Reference System in OGC URN format, e.g. `urn:ogc:def:crs:EPSG::4326`. Short-form EPSG codes (e.g. `EPSG:4326`) are also accepted. When omitted, the default is `urn:ogc:def:crs:EPSG::4326` (WGS 84 longitude/latitude). | +| `crs` | geometry, geography | Yes (geometry) / No (geography) | string | The Coordinate Reference System. Accepted forms: authority codes (e.g. `EPSG:4326`, `OGC:CRS84`) — **recommended**; OGC URN identifiers (e.g. `urn:ogc:def:crs:EPSG::4326`); an inline PROJJSON document (as a JSON string); a `srid:` SRID reference (e.g. `srid:0` for unspecified); or a `projjson:` reference to a PROJJSON blob stored elsewhere in metadata. Required for `geometry` — no universal default exists for planar coordinate systems. When omitted for `geography`, `EPSG:4326` (WGS 84) is assumed; note that GeoParquet 2.0 and Parquet native geospatial use the equivalent `OGC:CRS84` to make the (longitude, latitude) axis order explicit. | | `dimensions` | geometry, geography | No | integer | Number of coordinate dimensions: `2` (XY, default), `3` (XYZ or XYM), `4` (XYZM). | -| `algorithm` | geography only | No | string | Interpretation of edges between vertices. One of `spherical` (great-circle arcs on the unit sphere, default) or `vincenty` (geodesic on a reference ellipsoid). Ignored for `geometry`. | -| `encoding` | geometry, geography | No | string | The physical serialisation format of the geometry value. One of `wkt`, `wkb`, `geojson`, `ewkt`, `ewkb`. When omitted, the encoding is system-defined or unspecified. | +| `algorithm` | geography only | No | string | Interpretation of edges between vertices. One of `spherical` (great-circle arcs on a sphere, default), `vincenty` (Vincenty's formulae on an ellipsoid), `thomas`, `andoyer`, or `karney` (GeographicLib). Values align with Apache Parquet native geospatial and GeoParquet 2.0. Ignored for `geometry` (edges are always planar). | +| `encoding` | geometry, geography | No | string | The physical serialisation format of the geometry value. One of `wkb` (canonical for GeoParquet 2.0 and Parquet native geospatial), `wkt`, `geojson`, `ewkt`, `ewkb`. When omitted, the encoding is system-defined or unspecified. | +| `bbox` | geometry, geography | No | array | Bounding box of the column's spatial data, expressed in the column's own `crs` (matching GeoParquet 2.0). Format: `[xmin, ymin, xmax, ymax]` for 2D data; `[xmin, ymin, zmin, xmax, ymax, zmax]` when a Z dimension is present; `[xmin, ymin, zmin, mmin, xmax, ymax, zmax, mmax]` when both Z and M are present. Used as a spatial extent validation hint. | +| `orientation` | geometry, geography | No | string | Winding order for polygon rings. Currently only `counterclockwise` is defined (exterior rings counterclockwise, interior rings clockwise), matching GeoParquet 2.0. Recommended for `geography` with non-planar edges to avoid ambiguity around which side of a ring is "inside". | +| `epoch` | geometry, geography | No | number | Decimal year (e.g. `2021.47`) indicating the coordinate epoch for dynamic CRSs whose reference frames evolve over time. Optional; only meaningful when the `crs` is dynamic. | ### Example 1: Minimal — a GPS coordinate column @@ -136,14 +139,14 @@ schema: description: "Parcel boundary polygon in the Dutch RD New projection." logicalTypeOptions: subType: Polygon - crs: urn:ogc:def:crs:EPSG::28992 + crs: EPSG:28992 dimensions: 2 encoding: wkb - name: centroid logicalType: geometry logicalTypeOptions: subType: Point - crs: urn:ogc:def:crs:EPSG::28992 + crs: EPSG:28992 dimensions: 2 encoding: wkt ``` @@ -165,7 +168,7 @@ schema: description: "Full 3D flight path (longitude, latitude, altitude in metres)." logicalTypeOptions: subType: LineString - crs: urn:ogc:def:crs:EPSG::4326 + crs: EPSG:4326 dimensions: 3 algorithm: vincenty encoding: wkb @@ -184,22 +187,35 @@ The `physicalType` field carries the target system's native column type, while ` | Apache Iceberg v3 | `geometry` / `geography` | | DuckDB (spatial ext.) | `GEOMETRY` | | Apache Sedona | `geometry` | -| GeoParquet | `BYTE_ARRAY` (WKB, Parquet binary) | +| GeoParquet 2.0 | `GEOMETRY` / `GEOGRAPHY` (native Parquet logical types over `BYTE_ARRAY`, WKB) | | Oracle Spatial | `SDO_GEOMETRY` | | SQL Server | `geometry` / `geography` | ### Coordinate Reference Systems -The `crs` option accepts OGC URN identifiers as defined in OGC 07-092r3. Common values: +The `crs` option accepts several forms: -| CRS name | URN | Short form | -| ------------------------------- | --------------------------------------- | --------------- | -| WGS 84 (longitude/latitude) | `urn:ogc:def:crs:EPSG::4326` | `EPSG:4326` | -| WGS 84 / Pseudo-Mercator | `urn:ogc:def:crs:EPSG::3857` | `EPSG:3857` | -| Dutch RD New | `urn:ogc:def:crs:EPSG::28992` | `EPSG:28992` | -| UTM Zone 32N | `urn:ogc:def:crs:EPSG::32632` | `EPSG:32632` | +- **Authority codes** — e.g. `EPSG:4326`, `OGC:CRS84`. **Recommended.** +- **OGC URN identifiers** — e.g. `urn:ogc:def:crs:EPSG::4326`. +- **Inline PROJJSON** — a full PROJJSON document as a JSON string, for CRSs that cannot be referenced by an authority code. +- **`srid:`** — an SRID reference (e.g. `srid:0` for unspecified), aligning with Apache Parquet native geospatial. +- **`projjson:`** — a reference to a PROJJSON blob stored elsewhere in metadata. -When `crs` is omitted, `urn:ogc:def:crs:EPSG::4326` (WGS 84) is assumed, matching the GeoParquet and GeoJSON conventions. +Common values: + +| CRS name | Authority code (recommended) | OGC URN | +| -------------------------------------------------------- | ---------------------------- | ------------------------------ | +| WGS 84 (longitude/latitude, explicit lon/lat axis order) | `OGC:CRS84` | `urn:ogc:def:crs:OGC::CRS84` | +| WGS 84 (longitude/latitude) | `EPSG:4326` | `urn:ogc:def:crs:EPSG::4326` | +| WGS 84 / Pseudo-Mercator | `EPSG:3857` | `urn:ogc:def:crs:EPSG::3857` | +| Dutch RD New | `EPSG:28992` | `urn:ogc:def:crs:EPSG::28992` | +| UTM Zone 32N | `EPSG:32632` | `urn:ogc:def:crs:EPSG::32632` | + +`EPSG:4326` and `OGC:CRS84` refer to the same datum (WGS 84); they differ only in the conventional axis order (`EPSG:4326` is defined as latitude/longitude, `OGC:CRS84` as longitude/latitude). GeoParquet 2.0 and Parquet native geospatial use `OGC:CRS84` to make the axis order unambiguous. + +When `logicalType` is `geometry`, `crs` is **required**: there is no universal default for planar coordinate systems, and assuming one leads to data quality issues. + +When `logicalType` is `geography` and `crs` is omitted, `EPSG:4326` (WGS 84 longitude/latitude) is assumed, matching the GeoJSON convention and the historical ODCS default. For GeoParquet 2.0 interoperability, prefer `OGC:CRS84` explicitly. ### Geometry subtypes (ISO 19125-1) @@ -215,6 +231,56 @@ The `subType` option maps directly to the ISO 19125-1 Simple Features geometry h | `MultiPolygon` | A collection of polygons | | `GeometryCollection` | A heterogeneous collection of any geometry subtypes | +#### Mapping to GeoParquet 2.0 `geometry_types` + +GeoParquet 2.0 expresses the set of subtypes present in a column as an array of strings under `geometry_types`, with dimension suffixes baked into each string (` Z`, ` M`, ` ZM`). ODCS keeps `subType` (a single string, the union of allowed subtypes) and `dimensions` (an integer) as separate options, in line with the rest of `logicalTypeOptions`. The mapping is straightforward: + +| ODCS `subType` + `dimensions` | GeoParquet 2.0 `geometry_types` entry | +| ----------------------------------- | ------------------------------------- | +| `Point`, `dimensions: 2` | `Point` | +| `Point`, `dimensions: 3` (XYZ) | `Point Z` | +| `LineString`, `dimensions: 3` (XYM) | `LineString M` | +| `Polygon`, `dimensions: 4` (XYZM) | `Polygon ZM` | + +When `subType` is omitted (any subtype accepted), the corresponding GeoParquet 2.0 form is an empty `geometry_types` array (unknown/mixed types). + +### Spatial extent (bounding box) + +A bounding box can be declared at two levels to document and validate the spatial extent of geospatial data. Bounding boxes are always expressed in the CRS of the geometry they describe, matching GeoParquet 2.0 (whose bbox is stated in the column's own `crs`, not forced to WGS 84). + +#### Column-level `bbox` + +The `bbox` option in `logicalTypeOptions` records the expected spatial extent of an individual geometry or geography column, expressed in the column's own `crs`. Format: `[xmin, ymin, xmax, ymax]` for 2D data; `[xmin, ymin, zmin, xmax, ymax, zmax]` when a Z dimension is present; `[xmin, ymin, zmin, mmin, xmax, ymax, zmax, mmax]` when both Z and M are present. It serves as a validation hint: values falling outside the declared bounding box indicate data quality issues. + +```yaml +logicalTypeOptions: + subType: Polygon + crs: EPSG:28992 + bbox: [12621, 306846, 278026, 619256] # Netherlands in RD New (EPSG:28992) +``` + +When a dataset contains multiple spatial columns, each column carries its own `bbox`, which makes the per-column extent precise and unambiguous. + +#### Table-level `spatialExtent` + +A `spatialExtent` block at the schema (table) level captures the combined geographic footprint of the entire dataset. Because a table may combine columns with different CRSs, the table-level `spatialExtent.bbox` MUST carry its own `crs` alongside the array. When only one spatial column exists (or all spatial columns share a CRS), reusing that column's CRS is the natural choice. + +```yaml +schema: + - name: parcels + physicalName: cadastral_parcels + spatialExtent: + crs: EPSG:28992 + bbox: [12621, 306846, 278026, 619256] + properties: + - name: boundary + logicalType: geometry + logicalTypeOptions: + subType: Polygon + crs: EPSG:28992 + bbox: [12621, 306846, 278026, 619256] +``` + ### Relationship to existing types `geometry` and `geography` are not modelled as `string` or `binary` because: @@ -262,7 +328,7 @@ TBD. - **Non-breaking**: `geometry` and `geography` are new optional `logicalType` values; no existing contract is affected. - **Interoperable**: Maps cleanly onto PostGIS, BigQuery, Snowflake, Databricks/Iceberg v3, DuckDB, GeoParquet, and GeoArrow. - **Composable with other RFCs**: Works with [RFC-0034](0034-measures-and-dimensions.md) (geospatial columns can coexist with measures and dimensions) and [RFC-0041](0041-synonyms.md) (synonyms on geospatial columns aid discovery). -- **Validator impact**: Contract validators SHOULD warn when `logicalType` is `geometry` or `geography` and `logicalTypeOptions.encoding` is absent, as omitting it leaves the serialisation format ambiguous for consumers. +- **Validator impact**: Contract validators MUST warn (or error) when `logicalType` is `geometry` and `logicalTypeOptions.crs` is absent — no default exists for planar coordinate systems. Validators SHOULD warn when `logicalTypeOptions.encoding` is absent for either type, as omitting it leaves the serialisation format ambiguous for consumers. - **Binary support**: The original issue request for a binary logical type is addressed by combining `logicalType: geometry` (or `geography`) with `logicalTypeOptions.encoding: wkb`. ## References @@ -270,7 +336,8 @@ TBD. - [ISO 19125-1: Geographic information — Simple feature access — Part 1: Common architecture](https://www.iso.org/standard/40114.html) - [OGC 07-092r3: Definition identifier URNs in OGC namespace](https://docs.ogc.org/is/07-092r3/07-092r3.html) — specifies the `urn:ogc:def:crs:EPSG::*` URN format - [Apache Iceberg v3 spec — geometry and geography types](https://iceberg.apache.org/spec/#primitive-types) -- [GeoParquet specification](https://geoparquet.org/releases/v1.1.0/) +- [GeoParquet 2.0 specification (v2.0.0-rc.1)](https://geoparquet.org/releases/v2.0.0-rc.1/) +- [Apache Parquet native geospatial logical types (`GEOMETRY`, `GEOGRAPHY`)](https://github.com/apache/parquet-format/blob/master/Geospatial.md) - [GeoArrow specification](https://geoarrow.org/) - [PostGIS geometry/geography reference](https://postgis.net/docs/manual-3.5/using_postgis_dbmanagement.html#PostGIS_GeographyVSGeometry) - [Snowflake GEOGRAPHY data type](https://docs.snowflake.com/en/sql-reference/data-types-geospatial)