feat: replace QMesh terrain engine with direct GeoTIFF/COG sampling (MAPCO-11560)

[shimoncohen]

Summary

• Replace the Cesium quantized-mesh terrain engine with direct COG sampling read over HTTP range requests from the S3 gateway (geotiff.js). Cesium removed entirely.
• Positions flow as WGS84 degrees end-to-end; the Cesium radians round-trip (dataToRadians/dataToDegrees middlewares + radiansToOriginalPositionsMap) is deleted.
• Discovery still via CSW, now filtering GEOTIFF links (was TERRAIN_QMESH); DEMTerrainCacheManager opens one GeotiffHeightProvider per record (token auth, cogs/ object key).
• HeightsManager: per-point provider selection (productType + footprint + resolution), group-by-provider, bilinear sampling; nodata (-32768/NaN) and out-of-footprint → null. Per-record provider failures are isolated.
• /points API contract, response shape, and openapi3.yaml unchanged.

Feasibility verified live on OCP dem-dev: gateway honors Range (206) with OPA token auth; geotiff.js sampled a real MinIO COG (cogs/dtm_srtm30wgs84geo_tiled256_ovr_lzw.tif) end-to-end.

Test Plan

• npm run build clean
• npm run test:unit — 16/16, coverage passes
• npm run test:integration — 8/8 (incl. seeded happy-path exercising the real sampling chain), coverage passes
• No cesium/Cartographic/TERRAIN_QMESH/radians references in src/tests

Deployment (out-of-repo, before this serves heights)

• CSW DEM record(s) must expose a GEOTIFF link → .../cogs/<file>.tif in bucket dem-dev (today the live record has a TERRAIN_QMESH link to terrains/srtm100).
• Ensure the served object is the tiled_ovr COG (native 30m, tiled, overviews) — not the coarse _COG. Upload the SRTM100 COG if 100m coverage is needed.
• No helm change required (s3Gateway.url + accessToken already point at the internal gateway with queryParam token).

🤖 Generated with Claude Code

[shimoncohen]

Deployment TODO (ops) — required before this serves heights

This PR only changes the service. The following are data/infra steps and must land in dem-dev (and later stage/prod) for /points to return real elevations:

1. CSW record → GEOTIFF link. The DEM catalog record(s) must expose a link with protocol: GEOTIFF whose URL resolves (after the cogs/ split) to the object key in bucket dem-dev, e.g. https://tiles-dev.mapcolonies.net/api/dem/v1/cogs/dtm_srtm30wgs84geo_tiled256_ovr_lzw.tif. The live record today has a TERRAIN_QMESH link to terrains/srtm100 — that will no longer be discovered.
2. Serve the right COG. Use the tiled_ovr variant (native 30m, internally tiled 256, has overviews) — NOT the coarse _COG (resampled to ~38m). Confirmed on OCP: cogs/dtm_srtm30wgs84geo_tiled256_ovr_lzw.tif is present and range-readable.
3. SRTM100. No 100m COG under cogs/ yet — upload if 100m coverage is required.
4. No helm change required. Service reads s3Gateway.url (→ dem-nginx-s3-gateway-internal) and accessToken (queryParam token), both already set. Just confirm the cogs/ prefix is reachable through the internal gateway.

Verified live in dem-dev: gateway returns 206/Content-Range with OPA token auth, and geotiff.js opened + sampled the real MinIO COG end-to-end.

[shimoncohen]

Jira: MAPCO-11560

[syncush]

🔍 Detailed Code Review

Overall, this migration is clean and a great architectural improvement—it eliminates heavy Cesium dependencies, drops unnecessary radian/degree transformations, and streamlines direct COG reading over HTTP range requests.

Below is a detailed breakdown of findings, potential bugs, and suggestions before merging:

---

🚨 Critical / Major Finding: Input Point Order is Lost in Multi-Provider / No-Provider Scenarios

• File: src/heights/models/heightsManager.ts (Lines 639–681)
• Issue:
  samplePositionsHeights groups points into buckets based on providerKey:

  const groups = new Map<string | null, GeoPoint[]>();
  for (const position of positionsWithProviders) {
    const key = position.providerKey ?? null;
    const bucket = groups.get(key) ?? [];
    bucket.push({ longitude: position.longitude, latitude: position.latitude });
    groups.set(key, bucket);
  }
  ...
  const { results } = await PromisePool.for(groupEntries)
    ...
  finalPositionsWithHeights.push(...(results as PosWithHeight[][]).flat());

  Because points are bucketed by provider and then the bucket results are flattened with .flat(), the returned array does not preserve the original ordering of the input points list when the points span across multiple providers or include points outside of coverage (providerKey === null).
• Impact:
  Clients calling /points with an array [P1, P2, P3] (where P1 is in Provider A, P2 is out-of-bounds, P3 is in Provider A) will receive [P1, P3, P2] in the response instead of [P1, P2, P3].
• Suggested Fix:
  Preserve the original index with each position before bucketing, and sort/place results back into their original index slots before returning:

  // In attachProviderToPositions or samplePositionsHeights, keep track of originalIndex:
  type IndexedPoint = GeoPoint & { originalIndex: number };
  
  // When assembling finalPositionsWithHeights:
  const finalPositionsWithHeights: PosWithHeight[] = new Array(positionsArr.length);
  // Put each sampled point at its originalIndex

---

⚠️ Performance Consideration: Concurrency of readRasters per Provider

• File: src/heights/models/geotiffHeightProvider.ts (Lines 264–295)
• Observation:
  GeotiffHeightProvider.sample maps all points concurrently:

  public async sample(points: GeoPoint[]): Promise<(number | null)[]> {
    return Promise.all(points.map(async (point) => this.sampleOne(point)));
  }

  For large point batches (e.g. 500–1000 points in a single request), Promise.all triggers hundreds of simultaneous readRasters window calls. Even with internal block caching, uncached blocks will fire simultaneous HTTP range requests, potentially overwhelming gateway sockets or triggering connection limits.
• Recommendation:
  Consider bounding concurrency (e.g. using PromisePool or batching) or clustering points per TIFF tile block if high batch sizes are expected.

---

💡 Minor Improvements & Code Quality

1. Unused Injected Service in HeightsManager:

   • File: src/heights/models/heightsManager.ts (Line 574)
   • CommonErrors is injected into the constructor but is never referenced after removing POINTS_DENSITY_TOO_LOW_ERROR. Either remove it from DI or leave a short comment explaining it is retained for constructor backwards compatibility.

2. Query Parameter Construction:

   • File: src/heights/models/DEMTerrainCacheManager.ts (Lines 371–374)
   • String concatenation for query parameters (${objectUrl}${separator}...) could be replaced with the WHATWG URL API (new URL(objectUrl) / searchParams.set(...)) to safely handle existing params, query fragments, and URL encoding.

3. Coverage for Provider Initialization Failure:

   • File: src/heights/models/DEMTerrainCacheManager.ts (Lines 343–350)
   • initProviders wraps individual provider initialization in a try/catch to skip failed records, which is great. Adding a unit test verifying that one failing record does not prevent subsequent records from being registered would ensure this error resilience doesn't regress.

---

✅ Summary

Aside from the point-order preservation fix, the implementation is clean, well-tested, and achieves the goal of eliminating Cesium and switching to direct COG sampling.

[shimoncohen]

Addressed in 2dfc862:

• Bump minimist and commitizen #1 input order (critical): samplePositionsHeights now scatters sampled results back to each point's original index instead of flattening provider buckets. Added a regression test covering interleaved in-footprint / out-of-coverage points.
• Bump qs and formidable #2 concurrency: GeotiffHeightProvider.sample now bounds range reads via PromisePool with samplingConcurrency (config, default 16; env SAMPLING_CONCURRENCY).
• Bump json5 from 1.0.1 to 1.0.2 #3 unused CommonErrors: removed from HeightsManager DI.
• Bump cookiejar from 2.1.3 to 2.1.4 #4 query-param URL: buildAuthenticatedUrl now uses the WHATWG URL/searchParams API.
• Tests #5 init resilience: added a unit test asserting a failing record is skipped and the rest still register.

18 unit + 8 integration green, build clean.