decant

Lossless, reversible conversion between microscopy image files and OME-Zarr — with byte-exact reconstruction of the original file.

decant pours your image data into a compressed, natively-browsable OME-Zarr, and can pour it back out byte-for-byte identical to what went in. It fills a specific gap: existing converters (bioformats2raw, ngff-zarr, …) are only pixel-exact, and virtual-Zarr tools (kerchunk / VirtualiZarr) keep the originals byte-exact but don't compress them. decant does both — compress for space, reconstruct the exact original, keep native Zarr access.

Scope: this is the conversion/reconstruction tool only. Uploading to the cloud, scheduling, retention, and deletion are deliberately out of scope.

How it works

A TIFF is mostly pixel bytes wrapped in a little structure (header, IFDs, tags, padding). decant separates the two:

• pixels → an OME-Zarr array, recompressed with blosc + zstd + byte-shuffle by default (the benchmarked winner for 16-bit microscopy; codec + level are configurable);
• structure → a compact husk sidecar: everything that isn't pixel data, plus each pixel segment's offset/size.

Reconstruction re-derives each segment's bytes from the pixel array and splices them back into the husk at the recorded offsets — reproducing the original file exactly. Every pack is reconstructed and sha256-verified before the byte-exact claim is made; if it can't match (compressed input, exotic layout), decant falls back automatically. "Byte-exact" is over the file's content (its sha256) — filesystem metadata like mtime/permissions is out of scope.

Fidelity modes

Mode	When	Guarantee	Native OME-Zarr?
byte-exact	uncompressed TIFF	sha256(restored) == sha256(original)	yes
verbatim	compressed / non-TIFF (fallback)	byte-exact (zstd of the original)	no

Install

pip install -e .          # from a clone; needs numpy, tifffile, zarr<3, numcodecs

Quickstart

decant pack   image.tiff   image.zarr.zip   # -> one self-describing OME-Zarr bundle
decant pack   image.tiff   image.zarr.zip --pyramid            # + downsampled view levels
decant pack   big.tiff      big.zarr.zip --progress             # progress on large files
decant pack   image.tiff   image.zarr.zip --codec lz4 --clevel 1 --threads 4  # tune speed/ratio
decant check  image.tiff                    # predict pack mode + ratio (writes nothing)
decant verify image.zarr.zip                # reconstruct without writing to disk, check sha256
decant restore image.zarr.zip  out.tiff     # rebuild the exact original
decant restore series.zarr.zip one.tif --member frame_003.tif  # extract one series member
decant info   image.zarr.zip                # summary (mode, sizes, ratio, dims, codec)
decant info   image.zarr.zip --json         # the raw manifest

check triages a file (does it get the byte-exact path or fall back to verbatim, and why) without writing anything. When a real pack can't take the byte-exact path it falls back to verbatim and logs the reason on the "decant" logger.

Most archives are a single .zarr.zip file (a STORED ZipStore — one object per archive, not a directory of thousands of chunk files) carrying valid OME-NGFF multiscales metadata (axes, physical pixel sizes, channels); an OME-TIFF opens in napari/QuPath with its true dimensions, or in Fiji via the decant-fiji plugin. --pyramid adds derived downsample levels for fast zoomed-out viewing (~15–30% larger, never part of byte-exact restore). The pixel codec is configurable — --codec (zstd default, lz4, gzip, none, …) and --clevel — and the choice never affects byte-exactness (zarr stores the codec per array, so restore reads it back automatically).

import decant
res = decant.pack("image.tiff", "image.zarr.zip")   # res.mode == "byte-exact"
decant.restore("image.zarr.zip", "out.tiff")        # byte-identical
assert decant.verify("image.zarr.zip")

Status

Alpha (0.1.0), but broadly capable and well-tested (117 tests). Working today:

• Byte-exact mode-D for uncompressed TIFF — grayscale, RGB (chunky/planar), tiled (incl. non-zero edge padding), multi-sample, multi-strip; uint8/uint16/signed-int/float dtypes; little- and big-endian; classic TIFF and BigTIFF (validated on multi-gigabyte BigTIFFs) — with the verbatim fallback for anything else, always byte-exact.
• Single-file .zarr.zip archives (one object each; a heterogeneous multi-file series instead packs as a small directory of per-member bundles) with valid OME-NGFF 0.4 metadata (axes, physical pixel sizes, channels); OME-TIFF reshaped to its true N-D dimensions; an optional multiscale pyramid; a configurable pixel codec / level / threads.
• Multi-file series — auto-detected OME-TIFF, or an explicit list (with an axes/shape hint for loose frames) — consolidated into one stack and restored to the same N files byte-exact (single members extractable).
• Streaming in bounded memory (≈ one page + small buffers, not an enforced cap) regardless of file size.
• CLI + library: pack / pack-series / restore / verify (corruption-safe) / check (dry-run) / info, with a --progress readout and fallback-reason logging.
• A documented, versioned format spec (docs/format.md) — an archive is reconstructable from the spec alone (zarr + numpy), with no dependency on decant.

See DESIGN.md for the architecture and docs/format.md for the on-disk format.

License

MIT — see LICENSE.