Repository navigation
feat(examples): add a GPU hillshade of Mapterhorn PMTiles - #817
Merged
Merged
Conversation
…layers `DEFAULT_RANGE_READER`, a module-level `RangeReader` next to the default limiter, so every COG layer on a page shares one byte cache and one batching window. `COGSource.open`, `MultiCOGSource.open` and `MosaicLayer` take `reader` and `requestInit` (headers, credentials, mode for data behind authentication) and forward them to `GeoTIFF.fromUrl`; the mosaic passes them to `getSource` like the limiter. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…e task `COGLayer` and `MultiCOGLayer` default to `maxRequests: 256` and `debounceTime: 50`: deck.gl's six-at-a-time throttle would starve the range reader, which merges a selection's reads into few streamed requests and is capped by the per-origin limiter instead; the debounce lets a pan settle before anything is requested. `RasterTileLayer` and `MosaicLayer` keep deck.gl's defaults (#562). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`RasterTileset2D` gains `abortUnwantedLoads`: after every update, a loading tile that is neither selected nor visible is aborted. deck.gl does that only while more than `maxRequests` loads are in flight, which the COG layers' high `maxRequests` disables; without it a merged, streamed request would run on for tiles that left the view. `RasterTileLayer` exposes it as a protected hook, off by default; `COGLayer` and `MultiCOGLayer` turn it on. `ZarrLayer` keeps deck.gl's behaviour until its own reader adapter lands: zarrita's coalescer fails every read of a merged request when one aborts. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Requests, caching and authentication for the COG layers: the shared reader, tuning it or isolating a layer, `requestInit` for data behind authentication, and the new throttling defaults. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…x docs Findings of the whole-branch review: - deck.gl evicts unselected loading tiles inside `update` (its cache resize) before the tileset's own abort pass can see them, so a tile that left the view and fell out of the cache in one update kept its stream alive and later made a texture nobody would free. `RasterTileset2D` now wraps `onTileUnload` in `setOptions` (which `TileLayer` calls on every props change) to abort a loading tile first. Test added for both the option on and off. - The deck.gl-geotiff README showed `reader` and `requestInit` as `COGLayer` props, which do not exist: a TypeScript user got a compile error and a JavaScript user's auth header was silently dropped. The examples now open the source with `COGSource.open` and pass it to the layer; `MosaicLayer` takes both as props. - `finalize`'s docstring had been displaced above `update`. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Docstrings state what each option or hook is and the rule a caller needs; the reasoning behind the defaults and the abort rule is in the PR. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Designs an experimental example that does for PMTiles what cog-terrain does for COGs: Mapterhorn's terrain archive feeds both a TerrainLayer mesh and a RasterTileLayer drape that decodes Terrarium elevation on the GPU. Both read through the range reader, since a clustered archive keeps a view's tiles together: 20 tiles at the Matterhorn sit in 2 contiguous byte runs. Assumes question 10 of the Sources spec stays as it is. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rework the spec around #647's hillshade instead of 3D terrain: the same Horn kernel, halo and sun controls, fed by Mapterhorn's PMTiles through the range reader. The view and its halo ring (42 tiles) sit in 7 contiguous runs at the Matterhorn. Adds what Web Mercator needs (a ground cell size per fragment), a cache of tile borders with counted cancellation, MapLibre's exact pixel readback, and why the pmtiles library beats loaders.gl here. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Kyle settled four of the five open questions: no loaders.gl, this replaces #647 as the hillshade example (#647 is parked), the merge switch stays, and the pmtiles adapter lives in the example until it moves to library code. How a directly used RasterTileLayer aborts unwanted loads is still open. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Kyle chose a small RasterTileLayer subclass that overrides #809's protected _abortUnwantedLoads hook over a new public prop, for now. That closes the spec's last open decision. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The range reader now calls its structural purpose "metadata", and #807's chunkd adapter tracks validators. The PMTiles adapter follows both, and PMTilesSource.open takes the byte source so the merge switch can swap it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The PMTiles hillshade example needs a WebMercatorQuad tile matrix set with 512 px tiles (morecantile ships none), each tile's Mercator y range, and the ground cell size, which shrinks with latitude. The app itself is a placeholder map until the shading lands. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hillshade needs each tile's exact bytes, so tiles decode without premultiplied alpha or color conversion, and pixels are read back the way MapLibre reads DEM tiles: through a canvas, or through VideoFrame when the browser adds fingerprinting noise to canvas reads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ported from #647's halo, but each neighbor contributes only its border (its outermost rows and columns), which is all the ring needs and all a cache has to keep. Columns wrap across the antimeridian; rows past the poles replicate the tile's own edge. ringWrites gives the four strips that refresh just the ring once late neighbors arrive. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each tile and its eight neighbors share one load per tile through a cache that keeps only a settled tile's border (8 KiB, not the 1 MiB of decoded elevation). Cancellation is counted as pmtiles counts it for directories: a load aborts once every waiter's signal has aborted, so panning stops the halo reads for tiles that left the view. #647's cache let those run to completion. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A pmtiles Source backed by the range reader merges the reads of one task, such as a view's tiles once their leaf directory arrives, into a few streamed requests. Tile data reads use the "data" purpose and headers and directories "metadata". The bytes are copied out so a worker transfer can't detach the reader's cache. A changed validator raises pmtiles' EtagMismatch, on which pmtiles refetches its directories. PMTilesSource opens an archive as a RasterSource on a 512 px WebMercatorQuad grid. A counting FetchSource is the per-tile baseline the merge switch compares against. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Horn kernel, hillshade, slope, elevation and tinted-relief modules come from #647. TerrainDerivative changes in two ways: the ground cell size is computed per fragment from its Mercator y (one value per tile jumps 7% between z6 rows near 46°N), and the nodata substitution is gone, since Terrarium has no nodata value. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
getTileData asks for a tile and its eight neighbors in one synchronous pass, so their reads merge. It draws the tile as soon as it decodes, then rewrites only the halo ring when late neighbors land, as #647 did for the whole texture. The layer subclass turns on #809's protected abortUnwantedLoads hook, because the example raises maxRequests and deck.gl then stops aborting loads for tiles that left the view. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The hillshade app: Mapterhorn's planet.pmtiles opened through the range reader, four places from the Matterhorn to Yosemite, and #647's modes and sun controls. In headed Chrome the opening view loads in 7 HTTP requests, moving the sun refetches nothing, and there are no seams at tile edges, between tile rows at z5, or across the antimeridian. Also fixes a bug carried over from #647: on flat ground such as a lake, the shader took atan(0, 0), which GLSL leaves undefined, and the NaN drew lakes black in tinted relief. The aspect is now 0 there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A readout shows tiles drawn and decoded, HTTP requests and bytes, and a "Merge requests" switch swaps the range reader for pmtiles' own FetchSource and deck.gl's six-at-a-time default. Each opened archive gets its own reader, so both sides of the switch start cold. Measured at the Matterhorn: 7 requests merged against 33-35 per tile for the same ~5 MB, and the view completes about 7% sooner at 150 ms round trips. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The README explains what the example shows and how: merged reads through the range reader, the halo and its border cache, the latitude-aware cell size, exact decoding. It records the measured request counts and says what's provisional. The docs list it with the other experimental examples, with a card of the Matterhorn in tinted relief. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review found that deck.gl stops aborting a tile once it has drawn, so a
tile's neighbor reads ran on after it left the view, was evicted, or
the merge switch replaced the layer. They also ran on when the archive
had no tile to draw. The halo reads now hold their own signal, which is
aborted on those paths and by releaseHalo, which onTileUnload calls.
luma.gl keeps a texture's props, so createTexture({ data }) held each
tile's 1 MiB halo array for as long as the tile was cached. The array
is now written with writeData instead.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review found that the adapter's own validator tracking could leave pmtiles permanently unable to read a replaced archive. It threw on the read that first saw the new version, then accepted stale reads in flight, and could throw from the header re-read, which pmtiles caches as a failure. Use pmtiles' own check instead: it passes the header's etag with every later read, so throw EtagMismatch only when a response's validator differs from it. The header read carries no etag and always adopts the new version. A test drives a real PMTiles across the swap. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review found that the 250 ms stats poll re-rendered the whole app, and with it the layer. Each tick handed deck.gl's TileLayer a new renderTile, so it re-traversed and rebuilt every cached tile's sublayers, about four times a second while tiles loaded, the window the readout measures. The readout now polls on its own, and renderTile and the tile callbacks are memoized. Measured in headed Chrome: tileset setOptions calls during a pan load went from 4 to 0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review asked for a real check in Firefox with privacy.resistFingerprinting rather than the earlier simulation. Firefox 155 randomizes every canvas read and switches WebCodecs off, so decoding fell back to the canvas and drew random stripes as terrain. Unlike MapLibre, the decode now never falls back to an altered canvas: tiles fail with ExactPixelsUnavailableError, and the app shows one message in the panel. The README says so, and notes that a drawn tile's halo reads are cancelled only when it's unloaded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Measured again on the final code, after the readout stopped rebuilding the layer during loads: merged reads finish the opening view in 1.3-1.4 s against 1.6-1.9 s per tile on a fast connection, still with 7 requests against 33-35. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each tile used to fetch its eight neighbors too, so its halo was complete. That more than doubled the opening view's bytes (4.8 MB instead of 2.0 MB) and made the tiles in view wait behind the off-screen ring in the merged requests. Now a tile loads only itself. Its ring is filled from the borders of neighbors that have loaded, and a side with no loaded neighbor repeats the tile's own edge; that neighbor isn't in view, so neither is the side. When a neighbor loads later, the tile rewrites only its ring. TileBorders keeps loaded tiles' borders and tells loaded neighbors when one arrives; it replaces BorderCache, whose shared loads and cancellation are no longer needed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kylebarron
commented
Oct 28, 2026
Member
Author
There was a problem hiding this comment.
don't include the information modal in the screenshot
Member
Author
There was a problem hiding this comment.
Note
This reply was written by Claude (Claude Code), not by @kylebarron.
Recaptured without the panel in 6738ba1: the same Matterhorn view and size (1800 x 1013), with the panel hidden before the screenshot. The map's attribution stays in the corner, since Mapterhorn's terms ask for it.
🤖 Written by Claude Code
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
This PR was written by Claude (Claude Code), not by @kylebarron.
Adds
examples/experimental/pmtiles-terrain. It computes relief shading on the GPU from Mapterhorn's global terrain, read straight from one PMTiles archive byRasterTileLayer. The archive isplanet.pmtiles: 356 GB covering zooms 0-12, as Terrarium-encoded 512 px lossless WebP.It shows three things:
This becomes the hillshade example. It ports and reviews #647's Horn kernel, halo and modes; #647 stays parked. Ref #647.
The design is in
dev-docs/specs/2026-10-27-pmtiles-terrain-design.mdand the example's README.Built on the range reader
It reads PMTiles through
@developmentseed/range-reader(#806) and uses #809's protected_abortUnwantedLoadshook, both on main. Outside the example, the PR changes only the spec, one docs entry, the card image and lockfile additions.How it works
pmtileslibrary directly.@loaders.gl/pmtilestakes only a URL or aBlob, so its reads can't merge; it callsgetZxywithout a signal; and it pinspmtiles3.x.RangeReaderSourceimplementspmtiles'Sourceon the range reader.datapurpose; the header and directories usemetadata.pmtiles' ownetagcheck.PMTilesSource extends RasterSourcedescribes a 512 px WebMercatorQuad grid bounded to the archive.tilesetDescriptor; question 10 of the Sources spec stays as it is.RasterTileLayerlearns to take asource, this plugs in there.r32floattexture whose outer ring comes from its eight neighbors.TileBorderskeeps each loaded tile's border (8 KiB, not 1 MiB), so a tile still finds a neighbor's border after deck.gl has unloaded the neighbor.VideoFramewhen canvas reads are altered. Where neither gives exact bytes, tiles fail with a message in the panel instead of drawing wrong elevations.PMTilesTerrainLayerexists only to turn on feat(deck.gl-geotiff): coalesce COG tile reads by default #809's protected_abortUnwantedLoadshook, because the example raisesmaxRequests. It can go once a public prop exists.Measured
The opening view at the Matterhorn: 12 tiles.
The merged opening view is seven requests:
Mapterhorn serves over HTTP/2, so merging saves more requests than time. Decoding takes about 1.25 ms per tile on the main thread.
Halos without fetches. At first, each tile also fetched its eight neighbors for its halo. Kyle found the example slow, and filling halos only from neighbors that load anyway made it faster:
Verified
vite build. Before the halo change,deck.gl-raster(251) andrange-reader(99) also passed on feat(deck.gl-geotiff): coalesce COG tile reads by default #809's head.VideoFramepath.privacy.resistFingerprinting: canvas reads come back random andVideoFrameis undefined. Tiles fail with the panel message; before the fix they drew random stripes.Review
A fresh reviewer found four Important issues, and I raised two minors to Important. All are fixed, each verified by a failing test or browser check first.
pmtilesunable to read it until reload. Fixed by usingpmtiles' ownetagcheck.createTexture'sdataprop, so the array is now uploaded withwriteData. The COG layers do the same:render-pipeline.tsandmulti-cog-layer.ts.setOptionscalls during a pan load went from 4 to 0.Also found in the browser: #647's shader takes
atan(0, 0)on flat ground, and the resulting NaN drew lakes black. It's fixed here, but #647 still has it.Open questions
createTexture'sdata?pmtilescaches failed directory fetches, so a network blip blanks a region until reload. That's upstream behavior and isn't filed.Calls made without asking
rebase --onto.FetchSource,getUniforms(proven by a mutation), a released tile's refine, andgetZxyreturning nothing.TerrainDerivativeexplicit uniform and binding types.loadTerrariumElevationlives inget-tile-data.ts.ControlPanelhas noexperimentalprop on this base.map.triggerRepaint. Cost if wrong: a refined ring could show a frame late.nulltiles, andrenderTiledraws nothing for them.atan(0, 0)on flat ground.writeDatacalls each.Deferred minors
performance.measureentries pile up over a session.🤖 Written by Claude Code