Provide more builtin processes

[fmigneault]

Description

Similarly to openEO that defines a list of predefined processes, Weaver can offer Buitin Processes that can be used to define multiple thematic operations.

For now, these operations are fairly limited, since the standardized OGC API - Processes approach of "Bring your own Process" is employed, which defines "How to describe a process", but not "Which processes are required".

More commonly used thematic processes should be integrated to facilitate their adoption and use. Notably, using ad-hoc CWL workflows would be simplified if most of the common/reusable parts of a workflow were predefined, robust and tested across multiple situations.

The tools should support namespaced IDs to facilitate thematic grouping (similar to what other references do).
They should also rely on available tools (dockerized application of existing CLIs) because of how well they are typically optimized, battle-tested and known to the communities.

Tools

These are some examples of potential tools or source of inspiration to implement capabilities:

• CDO — The Climate Data Operator (CDO) software is a collection of many operators for standard processing of climate and forecast model data
  • https://mpim-sw.gitlab-pages.dkrz.de/cdo/alphabetic_list.html
• openEO Processes — thematic process schemas, I/O and descriptions
  • https://processes.openeo.org/
  • https://openeo.cloud/process-capabilties/
  • https://github.com/Open-EO/openeo-processes
• GDAL — translator library for raster and vector geospatial data formats
  • https://gdal.org/en/stable/user/band_algebra.html
  • https://gdal.org/en/stable/user/gdal_python_utilities.html
  • https://gdal.org/en/stable/programs/index.html#gdal-application
• Whitebox-Tools — GIS, remote sensing, terrain analysis, hydrology, and LiDAR processing
  • https://www.whiteboxgeo.com/manuals/pro/introduction.html
  • https://web.archive.org/web/20260226051654/https://www.whiteboxgeo.com/wte-tools-list/
• netCDF Kitchen Sink — ncks combines selected features of ncdump, ncextr, and the nccut and ncpaste specifications into one versatile utility
  • https://linux.die.net/man/1/ncks
• xclim — Climate service processes and indice computation
  • Already being considered through https://github.com/bird-house/goldfinch
  • https://xclim.readthedocs.io/en/stable/indicators.html
  • https://xclim.readthedocs.io/en/stable/indices.html
• gimi-tools — GeoTIFF (or any other really) to HEIF / GIMI image converter, tiling, etc.
  • dependencies: https://github.com/strukturag/libheif.git and rasterio
• rasterio — GEO-image manipulations
  • https://rasterio.readthedocs.io/en/stable/cli.html
• s2mosaic
  • Mosaic Sentinel-2 MGRS tile creation
  • Various combination methods, bands, AOI/TOI
  • Auto discovery of tiles via public Microsoft Planetary Computer or Element 84's Earth Search on AWS
  • Cloud-Mask via OmniCloudMask deep-learning model on R+G+NIR bands or via L2A Scene Classification Layer
• supermorecado
  • TileMatrixSet, Quads, GeoJSON
  • Tiles Edge/Merge/Conversions
• geolibre-rust
  • combines 700+ CLI tools from whitebox_next_gen (similar to above whitebox-tools) and GeoLibre
  • tools: https://opengeos.org/geolibre-rust/
  • demo: https://www.youtube.com/watch?v=W32bIQO_nG8
• pyramids-gis (not to be confused with pyramid API framework used by this repo!)
  • GIS utility package built on top of GDAL/OGR for working with raster data (GeoTIFF, NetCDF), vector data (shapefiles, GeoJSON), and multi-temporal datacubes.
  • https://www.linkedin.com/posts/mafarrag_gis-python-remotesensing-share-7472046802129629184-Ddwf

References

• Support ad-hoc CWL workflows #834
• Support namespaced process ID #837

[fmigneault]

@huard @mishaschwartz @QSparks
FYI issue to track tools to be considered.