Skip to content

Replace Fortran extension _pytesselate with a Python implementation - #575

Open
gmloose wants to merge 3 commits into
pure-python-implementationfrom
replace-pytesselate
Open

gmloose wants to merge 3 commits into
pure-python-implementationfrom
replace-pytesselate

Conversation

@gmloose

@gmloose gmloose commented Nov 8, 2026 •

Copy link
Copy Markdown
Collaborator

Replaces the Fortran extension _pytesselate (f2py) with a pure-Python module, bdsf/tessellate.py, with tessellate_simple and tessellate_roundness, which have the same interface as pytess_simple and pytess_roundness. In psf_vary.py only the import (from . import tessellate) and the two calls change.

How it works

  • Pruning: the Fortran code computes the distance of every pixel to every generator. Here the image is divided into blocks of 128 × 128 pixels, and lower and upper bounds of the distances select the generators that can be the nearest one (or within the fuzzy margin) for any pixel of a block. Most blocks have a single candidate. The others are divided into blocks of 8 × 8 pixels, and the selection is repeated.
  • Exact arithmetic: for the small blocks that still have several candidates, the distances are computed with exactly the same floating point operations as the Fortran code (1-based coordinates, / (1 / (r * roundpix)) with two roundings, first strict minimum, ignoring NaN and distances ≥ 1e90). The bounds have a relative margin of 1e-9, far above the rounding errors. Every pixel gets the same value as in Fortran, bit for bit.
  • roundness: the tile centroids and pixel counts (integer sums, exact) are collected during the first tessellation. The order-dependent sum of the distances to the centroids is done in the order of the Fortran loop (N.bincount adds sequentially). The memory of the first tessellation is reused for the second.
  • Memory: no n × m integer array (Fortran: minind), so the peak is the output plus small temporaries.

Deviations (undefined behaviour in Fortran)

  • tessellate_roundness with fuzzy tessellation (psf_tess_sc='c'): the Fortran code uses the fuzzy result (sums of tile numbers) as array indices, which writes out of bounds. It crashed (munmap_chunk(): invalid pointer, double free or corruption) for both the test image and NGC6048. Here the tile properties are computed from the normal tessellation. PyBDSF doesn't process fuzzy tessellations, and the default is 's'.
  • Pixels without any valid generator (only for invalid input) and calls without generators give 0. In Fortran they were uninitialized.

Build

  • Removes src/fortran/pytess_simple.f, pytess_roundness.f and constants.h, the f2py module and find_package(F2PY) from CMake, the sources from MANIFEST.in, and meson from the build requirements (needed only by f2py).
  • Fortran is still needed for MINPACK and PORT3 (_cbdsm). The unused src/fortran/gaul2gaulbin.f is left alone.

Results

On a 20-core i7-12700 with NumPy 2.4.6, against the _pytesselate of a build of master. All results are identical to Fortran.

Single call Generators Fortran Python Peak MB (F / P)
test image 129², simple 's' 7 0.31–0.32 ms 0.26–0.29 ms 0 / ≤ 1
test image 129², roundness 's' 7 0.95 ms 0.83 ms 0 / 1
test image 129², simple 'c' 7 0.81–0.84 ms 0.39–0.44 ms 0 / 1
NGC6048 5762², simple 's' 15 1.36–1.38 s 0.12–0.15 s 379 / 255
NGC6048 5762², roundness 's' 15 3.84 s 0.57 s 379 / 255
NGC6048 5762², simple 'c' 15 3.73–3.77 s 0.25–0.36 s 379 / 256
random 5762², simple / roundness 50–200 4.3–51 s 0.16–1.0 s 379 / ≤ 255
random 2000², simple / roundness 1000 10–30 s 0.09–0.30 s 46 / ≤ 33

End-to-end (run_comparison.sh):

  • test/tbdsf_process_image.py passes, and all 48 FITS outputs are identical to master.
  • NGC6048 with psf_vary_do=True: all 44 FITS outputs (including volrank and the PSF maps) are identical, for simple and for roundness tessellation. The tessellation takes 0.12 s instead of 1.21 s (simple), and 0.50 s instead of 3.40 s (roundness).

Testing

  • New test/ttessellate.py, also added to CI and to the cibuildwheel test-command. It compares, exactly, with a direct transcription of the Fortran loops (itself checked against _pytesselate in 238 cases), with several block sizes:

    • random generators, all weightings, 's' and 'c'
    • ties and duplicate generators
    • empty tiles and tiles of one pixel (roundness)
    • invalid weights and positions
    • tiny, elongated and empty inputs
    • the interface
    • no floating point warnings or errors, also with seterr(all='raise')
  • benchmark/replace-pytesselate/ contains:

    • scripts to record the tessellation calls of a run and to benchmark single calls (results, speed, peak memory)
    • an end-to-end comparison that checks every FITS output exactly

    Details are in its DESCRIPTION.md.

  • A wheel built from a clean checkout (pip wheel ., no f2py) contains tessellate.py and no _pytesselate; tmulti_proc.py and ttessellate.py pass with it. On my machine, tbdsf_process_image.py crashes in _cbdsm with such a freshly built wheel, but exactly the same happens with a wheel of master built the same way, so it is unrelated.

🤖 Generated with Claude Code

@gmloose gmloose self-assigned this Nov 9, 2026
@gmloose
gmloose changed the base branch from master to pure-python-implementation November 9, 2026 12:29
@gmloose
gmloose requested a review from aroffringa November 9, 2026 12:39
@gmloose

gmloose commented Nov 9, 2026

Copy link
Copy Markdown
Collaborator Author

@codex

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-09T12:43:42.187608Z 4aa8f57 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: 4aa8f57751

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

gmloose and others added 2 commits November 9, 2026 17:29
Add `bdsf/tessellate.py`, which provides `pytess_simple` and
`pytess_roundness` with the same interface and bit-identical results.
Instead of computing the distance of every pixel to every generator,
the image is divided into blocks, and distance bounds select the
generators that can matter for each block. Only for small blocks on
tile boundaries are the distances computed, with the same floating
point operations as the Fortran code.

Remove the Fortran sources, the f2py module from CMake and MANIFEST.in,
and `meson` from the build requirements. Add `test/ttessellate.py` to CI
and the cibuildwheel tests, and benchmark scripts and results in
`benchmark/replace-pytesselate`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Failed runs of `tbdsf_process_image.py` and `run_psf_vary.py` were only
reported, and `compare_fits.py` succeeded without any FITS files to
compare. Now each failed step is recorded, and the script lists them and
exits with status 1. `compare_fits.py` fails if neither directory
contains FITS files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gmloose
gmloose force-pushed the replace-pytesselate branch from 4aa8f57 to 2f8629c Compare November 9, 2026 16:30
`pytess_simple` and `pytess_roundness` in `bdsf/tessellate.py` are now
`tessellate_simple` and `tessellate_roundness`, and `psf_vary.py` uses
`from . import tessellate` instead of aliasing it to `_pytess`. The
benchmark scripts map the Fortran names to the new ones, and
`record_tess.py` works with either version of `psf_vary.py`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant