VDiskProbe

VDiskProbe is a dependency-free Python library and CLI for detecting virtual disk image formats and reading available size metadata. It accepts local paths, binary file objects, custom random-access readers, and HTTP/HTTPS URLs.

VDiskProbe supports Python 3.11–3.15. Detection reads headers and metadata; it does not mount, convert, decrypt, or validate an entire image.

Architecture

The distribution, import package, and command are all named vdiskprobe. The package re-exports its public API from vdiskprobe/__init__.py. Detection follows this flow:

reader/path -> ByteSource -> DetectorRegistry -> ImageInfo

ByteSource adapts readers, DetectorRegistry selects a detector, and ImageInfo holds its result. Detector classes live in vdiskprobe/detectors/formats.py; binary parsing helpers live in vdiskprobe/detectors/_format_helpers.py.

With Python's default import policy, Python 3.15 loads built-in detector implementations and selected HTTP dependencies on first use. Python 3.11–3.14 load them eagerly. The public API is the same on all supported versions, and VDiskProbe leaves global import settings to the application. See Import compatibility for implementation and testing guidance.

Installation

Install from PyPI using a supported Python interpreter:

python -m pip install vdiskprobe

To install from a source checkout instead:

python -m pip install .

A built wheel or source archive can also be installed with python -m pip install <artefact-path>.

Formats and metadata

Built-in formats, in detection order: qcow, qcow2, qed, vmdk, vdi, vpc (alias vhd), vhdx, luks, bochs, parallels, cloop, dmg, and raw.

Results include the format, description, confidence, detector, aliases, source name, format-specific details, and these optional byte counts:

Field	Meaning
file_size	Logical source length.
disk_size	Host filesystem allocation when available.
virtual_size	Guest-visible size derived from image metadata.
cluster_size	Allocation unit or block size recorded by the format.

Unknown values are None in Python and null in JSON. On systems without filesystem block counts, disk_size falls back to logical file size. The content length of a remote image does not indicate how much disk space it occupies on the server.

raw is a fallback with confidence 0.1, meaning no recognised signature matched. It is not proof of a raw image. VMDK descriptor and cloop script heuristics use 0.95; other recognised signatures use 1.0.

VMDK extent sector counts that exceed the interpreter's configured decimal-conversion limit yield an unknown virtual size rather than a partial total. If that limit is disabled, VDiskProbe uses the interpreter's default limit. Descriptor capacity is also unknown when the source is known to extend beyond the bytes read, or when a descriptor of unknown length fills the 64 KiB inspection window. After a shorter read from a source of unknown length, VDiskProbe checks for additional data before reporting capacity.

Recognition can succeed on a short header without optional metadata and does not prove that the image is complete or uncorrupted. The read_only field describes format metadata; VDiskProbe itself only reads.

Command line

vdiskprobe disk.qcow2
vdiskprobe disk.qcow2 --json
python -m vdiskprobe disk.qcow2 --json
vdiskprobe --list-formats
vdiskprobe --version

A path may also be an HTTP/HTTPS URL. -v enables info logging, -vv enables debug logging, and -q selects errors only and takes precedence over verbosity. Results go to stdout and diagnostics to stderr. Exit statuses are 0 for success, 1 for source errors, and 2 for invalid arguments. --list-formats takes precedence over inspection and --json.

Python API

from pathlib import Path
from vdiskprobe import detect, detect_path

info = detect_path(Path("disk.qcow2"))
print(info.format, info.virtual_size)
print(info.to_dict())

with open("disk.qcow2", "rb") as stream:
    info = detect(stream, name="disk.qcow2")

detect and inspect_source accept a reader or an existing ByteSource, plus an optional registry and name. detect_path accepts a text path, PathLike[str], or HTTP/HTTPS URL; detect_url accepts only HTTP/HTTPS URLs. Use io.BytesIO(payload) for in-memory bytes.

Seekable readers retain their cursor. Caller-owned readers remain open; files and HTTP responses opened by the library are closed after use. A reader should expose stable contents during inspection. Sharing a reader between threads or modifying the source during inspection is not guaranteed to be safe.

Custom readers

Provide read_at(offset, size), pread(offset, size), read_range(offset, size), seek/tell/read(size), or read(offset, size), in that precedence order. Here pread takes the offset first, unlike os.pread.

from vdiskprobe import detect

class RangeReader:
    def __init__(self, payload):
        self.payload = payload

    def read(self, offset, size):
        return self.payload[offset:offset + size]

    def size(self):
        return len(self.payload)

info = detect(RangeReader(b"QFI\xfb\x00\x00\x00\x03"))
assert info.format == "qcow2"

Optional size sources are size or length attributes or methods, len(reader), a file descriptor, or seeking to the end of the source. ByteSource.read rejects negative offsets and sizes and returns at most the requested bytes. Reads up to 64 KiB can reuse cached prefixes and exact ranges. A short read caches only the bytes returned; it does not establish the end of the source.

Binary readers are preferred. At the adapter boundary, bytearray and memoryview objects are converted to bytes, None means an empty read, and text is encoded as Latin-1 without replacing unencodable characters. tobytes()/tostring() providers and conversion of other objects to text are also supported. Unencodable text raises an encoding error. Backend authentication, application-level retries, and SDK integration remain the caller's responsibility.

HTTP and HTTPS

from vdiskprobe import detect_url

info = detect_url("https://example.com/disk.qcow2")
print(info.format, info.virtual_size)

URL paths must be percent-encoded where necessary. URL encoding failures raise RemoteSourceError without retrying.

For explicit timeout and retry settings, construct HttpRangeReader from vdiskprobe.http_source and pass it to detect(reader).

HTTP requests use bounded byte ranges and identity encoding. By default, each request allows up to three retries after transient failures, for at most four attempts, with exponential backoff beginning at 0.25 seconds. Remote size discovery uses a one-byte GET before falling back to HEAD. A response that specifies a byte interval inconsistent with the request raises RemoteSourceError. If range metadata is absent or the total size is unknown, size discovery can still fall back to other response headers or a HEAD request.

A server that ignores ranges can still support a header match at offset zero. A refused range lets detection continue to later signatures; unavailable optional metadata remains unknown. If no signature matches, the refused range raises RangeNotSupportedError (a RemoteSourceError subclass in vdiskprobe.errors) rather than producing a raw result. Other transport failures propagate immediately. VDiskProbe never downloads the full image to simulate random access. Footer-based detection also depends on discovering source length.

TLS certificate and hostname verification stay enabled. Trust-store discovery checks SSL_CERT_FILE, REQUESTS_CA_BUNDLE, CURL_CA_BUNDLE, and SSL_CERT_DIR, then Python/OpenSSL defaults and common system paths. Redirects and proxy settings from the environment follow the standard library’s behaviour.

Custom detectors, errors, and logging

from io import BytesIO
from vdiskprobe import FormatDetector, build_default_registry, detect

class ExampleDetector(FormatDetector):
    format_name = "example"
    description = "Example image"
    priority = 5

    def detect(self, source):
        if source.read(0, 4) == b"EXMP":
            return self.build_info(source)
        return None

registry = build_default_registry()
registry.register(ExampleDetector())
info = detect(BytesIO(b"EXMP"), registry=registry)
assert info.format == "example"

Lower priorities run first; equal priorities retain registration order. The first match wins. Keep raw as the final fallback when creating a registry yourself; if no detector matches, the registry raises RuntimeError. Source failures propagate rather than becoming a raw result; refused HTTP ranges follow the bounded detection behaviour described above.

InvalidSourceError and RemoteSourceError inherit from VDiskProbeError. Local I/O failures can raise OSError.

VDiskProbe leaves logging configuration to the application. By default, its loggers use NOTSET and propagate records to ancestor handlers, inheriting the effective logging level from the logger hierarchy. The attached NullHandler produces no output and does not prevent propagation. Applications that already configure root logging normally need no VDiskProbe-specific setup.

For dedicated console output in a standalone script, call configure_logging() to set the package logger's level and attach a stream handler. This disables propagation to avoid also sending records to ancestor handlers. It does not configure the application's root logger.

import logging
from vdiskprobe import configure_logging

configure_logging(logging.DEBUG)

To explicitly override the package logging level while using the application's existing ancestor handlers, use configure_logging(logging.DEBUG, propagate=True). This sets the package logger's level to DEBUG instead of inheriting it. Ancestor logger levels do not filter propagated records, but handler levels and filters still apply. To restore level inheritance, use configure_logging(logging.NOTSET, propagate=True).

Enabling propagation removes the package stream handler and null handlers while preserving other application-owned handlers; it adds no stream handler. Repeated stream configuration reuses an existing package stream handler. An explicit stream replaces its destination, even if the previous stream has closed; omitting it or passing None preserves the destination. A new handler defaults to stderr. VDiskProbe does not close caller-owned streams. Applications can also configure the package logger directly with logging.getLogger("vdiskprobe"). get_logger(name) is intended for package modules. Applications using the library should use their own logging.getLogger(__name__).

Development

Run commands from the repository root with a supported Python interpreter. Use venv/ as the project's development environment and configure your IDE to use its interpreter. Reuse it if it already exists; otherwise, create it before activating it and installing the development tools:

python -m venv venv
source venv/bin/activate
python -m pip install -e '.[dev]'

Use the activation command appropriate for your shell. Keep one primary development environment; separate temporary environments are still needed for compatibility and clean-install checks. Test, formatting, and build dependencies belong in the dev extra in pyproject.toml; upload tooling is installed separately as described under Releasing.

Required checks

After every Python change, all four checks must pass: Black formatting, a limit of fewer than 1000 lines per Python file, the complete test suite with 100% source statement and branch coverage, and Git whitespace validation. A failed or unavailable check means verification is incomplete.

Black reads its settings from pyproject.toml. Format the Python files, then check them:

python -m black vdiskprobe tests
python -m black --check vdiskprobe tests

Check every tracked or unignored Python file, including tests and scripts. This prints files with 1000 or more lines and exits with a non-zero status if any exist:

python - <<'PYTHON'
from pathlib import Path
import subprocess

paths = subprocess.check_output([
    "git", "ls-files", "--cached", "--others", "--exclude-standard", "-z", "--", "*.py"
]).decode().split("\0")
failures = []
for name in sorted(set(paths)):
    if name and Path(name).is_file():
        count = len(Path(name).read_text(encoding="utf-8").splitlines())
        if count >= 1000:
            failures.append(f"{name}: {count} lines")
print("\n".join(failures), end="\n" if failures else "")
raise SystemExit(bool(failures))
PYTHON

The default pytest command enforces the coverage gate from pyproject.toml, measuring only vdiskprobe. Tests use synthetic disk headers, injected readers, and a loopback HTTP server. Keep assertions meaningful; coverage alone does not prove correctness.

python -m pytest
git diff --check

For a focused test run without the whole-package coverage gate:

python -m pytest --no-cov tests/test_detection.py
python -m pytest --no-cov tests/test_detection.py -k qcow2

Focused runs do not replace the required full-suite check. When coverage fails, inspect the missing paths in HTML or export XML for tooling:

python -m coverage html
python -m coverage xml

Open htmlcov/index.html for the HTML report. Both report commands retain the configured 100% threshold and can exit with a non-zero status while still writing their reports.

Builds and compatibility

Build a wheel and source distribution, then run the explicit packaging checks:

python -m build
python -m pytest -m packaging -o addopts='' tests/test_packaging.py

Packaging tests are excluded from the default suite. They inspect distribution contents and metadata, check test collection from the extracted source archive, install both artefacts in separate clean environments, and exercise imports and both CLI entry points outside the checkout. pip must be able to obtain the build dependencies, either from a package index or a local wheel cache.

Repeat the full suite and packaging checks on each supported Python minor version when changing compatibility or packaging. Support tracks stable Python releases that have not reached end of life; check the Python release lifecycle when updating the supported range. Keep syntax and dependencies compatible with the minimum version listed above.

Testing currently covers standard CPython builds. Other Python implementations and free-threaded builds remain unverified.

Import compatibility

Keep import syntax compatible with Python 3.11. To make selected imports lazy under PEP 810, define a __lazy_modules__ set containing fully qualified module names before the relevant ordinary imports. Do not use the Python 3.15-only lazy import syntax. The declaration applies only within the module that defines it. Accessing an imported name during module initialisation can load the dependency immediately, so verify loading behaviour in a fresh interpreter. Leave global lazy-import settings and filters to the application.

Changes to import behaviour require the full suite and packaging checks on every supported Python minor version. tests/test_imports.py checks eager imports on older Python versions, deferred loading, first use, public exports, CLI behaviour, and annotation resolution in fresh subprocesses. Measure startup and first-use performance against the unchanged revision. Also check byte reads, parsing, and detector dispatch after initialisation for regressions. Alternate repeated measurements of the original and changed code using the same interpreter and inputs, and report the environment and variability with any performance claim.

Versioning

Versions must follow PEP 440, for example 0.1.0, 1.0.0a1, or 1.0.0.dev1. The single source of truth is [project].version in pyproject.toml. vdiskprobe/version.py reads the installed distribution metadata using importlib.metadata, and the package re-exports that value as __version__. Install the package before importing it; an editable development install is sufficient. After changing the version, rerun python -m pip install -e '.[dev]' to refresh editable-install metadata, or rebuild and reinstall the distribution.

Releasing

These instructions are for authorised release maintainers. Token-based publishing requires Owner or Maintainer access to the vdiskprobe project on the destination index and an API token whose scope includes the project. TestPyPI and PyPI permissions are managed separately; GitHub repository access does not grant publishing access. Contributors can build and test locally without publishing access. See PyPI's project roles.

Record release changes in CHANGELOG.md and validation results in the release PR. Keep account-specific setup notes outside the repository and credentials in a password manager or credential store.

Prepare and validate

1. Set the intended version in pyproject.toml and finalise its dated changelog entry. For an early TestPyPI rehearsal, use a development version such as 1.1.0.dev1; prepare and validate the final version separately.
2. Refresh the editable installation as described under Versioning. Run all required checks and the packaging checks. Repeat the full suite and packaging checks on every supported Python minor version when changing compatibility or packaging. Run builds, editable installs, and tests sequentially: builds can refresh local package metadata while tests are reading it.
3. Merge the release PR from dev into main. Check out the clean release commit, confirm its version and changelog, and create and push the matching v<version> tag. A tag or GitHub Release does not upload anything to PyPI.
4. Build the final artefacts from that commit into a new directory. The following commands use a POSIX shell and an activated development environment; run them from the repository root:

python -m pip install --upgrade twine
release_version=$(python -c 'import tomllib; print(tomllib.load(open("pyproject.toml", "rb"))["project"]["version"])')
release_dir="$PWD/dist/release-$release_version"
test ! -e "$release_dir" && python -m build --outdir "$release_dir"
python -m twine check --strict "$release_dir"/*

Stop if a command fails. If the directory already exists, inspect it before proceeding; do not mix old and new builds. Twine is an upload tool installed separately from the development extra. Install the final wheel and source archive in separate clean environments and exercise imports and both CLI entry points outside the checkout, as the packaging checks do. Retain these exact files for both uploads; rebuild and revalidate if release content changes.

Local artefact retention

Keep only the latest final release build locally, in dist/release-<version>/. After that build succeeds, remove older release artefacts and superseded development, rehearsal, and validation builds from dist/ and repository-local scratch directories. Keep validation records separately from the artefacts. Preserve the latest wheel and source archive unchanged through validation and both uploads. This cleanup applies only to local build files, not Git tags, GitHub Releases, or files published to package indexes.

Rehearse on TestPyPI

TestPyPI and PyPI have separate accounts and API tokens. Use the token for the destination index, with __token__ as the username. Enter tokens at the private password prompt or use a credential store; never put them in commands, documentation, or Git. See the TestPyPI guide and PyPI authentication guidance.

python -m twine upload --repository-url https://test.pypi.org/legacy/ \
  --username __token__ "$release_dir"/*

Install the uploaded version in a fresh environment outside the checkout. This example uses the wheel; local packaging checks also cover source installation. --no-deps is appropriate because VDiskProbe has no runtime dependencies.

smoke_dir=$(mktemp -d)
python -m venv "$smoke_dir/venv"
(
  cd "$smoke_dir" || exit
  unset PYTHONPATH
  "$smoke_dir/venv/bin/python" -m pip install --no-cache-dir \
    --index-url https://test.pypi.org/simple/ --no-deps "vdiskprobe==$release_version"
  "$smoke_dir/venv/bin/vdiskprobe" --version
  "$smoke_dir/venv/bin/python" -m vdiskprobe --list-formats
  "$smoke_dir/venv/bin/python" -c 'from io import BytesIO; from vdiskprobe import detect; assert detect(BytesIO(b"QFI\xfb\x00\x00\x00\x03")).format == "qcow2"'
)

Check that the version matches, all 13 formats are listed, and detection succeeds. Inspect the TestPyPI project description and uploaded files. Compare the files' SHA-256 digests from the index's release JSON endpoint (https://test.pypi.org/pypi/vdiskprobe/<version>/json) with the retained local artefacts.

Publish and verify

After the final version passes the TestPyPI rehearsal, upload those same artefacts to PyPI using the production token:

python -m twine upload --repository-url https://upload.pypi.org/legacy/ \
  --username __token__ "$release_dir"/*

Repeat the smoke-test block with a new temporary environment and https://pypi.org/simple/ as the index. Verify the description, version, wheel, source archive, and SHA-256 digests against https://pypi.org/pypi/vdiskprobe/<version>/json. Publication is complete once these checks pass.

Uploaded filenames cannot be reused, even after deletion; corrections to a published release need a new version. If an upload is interrupted, inspect the index and compare hashes before uploading only a missing file. Do not use --skip-existing to hide an unexplained mismatch. See PyPI's filename reuse policy.