Skip to content

fix: pick tools keep identity and place particles at their centre (v2.0) - #108

Merged
uermel merged 4 commits into
v2.0from
uermel/pick-identity-v2
Oct 6, 2026
Merged

fix: pick tools keep identity and place particles at their centre (v2.0)#108
uermel merged 4 commits into
v2.0from
uermel/pick-identity-v2

Conversation

@uermel

@uermel uermel commented Oct 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Summary

Part 1 of 3 of filament support in copick-utils, v2.0 line (cherry-picked twin of the main PR). Stack (v2.0): #108 → #109 → #110; main twins #105 → #106 → #107. CLI docs: copick/copick#496.

Requires copick 2.0.0a3, which ships the v2.0 filament stack (copick/copick#481, #482, #493, #495). The first commit built against the unreleased stack and the last one sets the floor, so squashed, the dependency change is copick>=2.0.0a3,<3, with uv.lock resolving it from PyPI.

The pick tools now treat a pick as the particle it describes:

  • picksin, picksout and clippicks test each particle's centre (location plus the transform's translation) and write the surviving points unchanged: transform, instance ID, score and order. An empty result writes an empty pick set instead of failing and leaving stale output behind.
  • clippicks --ref-seg indexed a (Z, Y, X) volume with (X, Y, Z) indices; it now indexes it correctly, also for non-cubic volumes.
  • picks2seg and the pick-to-mesh converters (picks2sphere, picks2ellipsoid, picks2plane, picks2surface, picks2mesh) place each particle at its centre.
  • Shared helpers in util/picks.py.

Tests

tests/test_pick_tools.py: identity and order survive the filters, a shifted particle is filtered and painted at its centre, a non-cubic --ref-seg, an empty result. Each fails on v2.0. The suite passes locally (68 tests).

🤖 Generated with Claude Code

This was referenced Oct 6, 2026
Merged
Merged
Merged
Merged
Merged
@uermel
uermel added this pull request to stack #114 October 6, 2026 09:07
@uermel
uermel marked this pull request as ready for review October 6, 2026 09:07
uermel and others added 4 commits October 6, 2026 02:13
@uermel @claude
chore: build against the copick filament stack (drop at release)
e071cc0 
copick-utils' filament, instance and pick-identity work uses APIs from the open
copick v2.0 PRs #481, #482, #493 and #495 (filament objects, pick identity, the
Filaments entity with curves, instance and panoptic segmentations). Until
those are released, depend on their stack's tip and lock it. Replace this with
a version floor once copick publishes them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@uermel @claude
fix: pick tools keep identity and place particles at their centre
5da0f99 
picksin, picksout and clippicks rebuilt their output from picks.numpy()/
from_numpy(), which dropped every pick's instance_id and score. For a filament
object that loses which filament a pick belongs to. They also tested the
pick's location, not its particle centre: copick defines the centre as the
location plus the transform's translation (a shift, e.g. from a RELION
import). Now they test the centre and copy the surviving points unchanged and
in order, so orientation, shift, instance_id and score survive.

Also:
- An empty result writes an empty pick set. Before, the run was reported as an
  error and an earlier output in the same place was left on disk.
- An instance segmentation works as a reference (any non-zero ID is inside),
  and a panoptic one is tested on its label channel.
- clippicks --ref-seg built its distance field on the segmentation's (z, y, x)
  grid but indexed it with (x, y, z) indices, which is wrong for any non-cubic
  volume. It now indexes (z, y, x) and rounds to the nearest voxel, matching
  copick's voxel-to-Angstrom mapping (voxel i at i * spacing).
- picks2seg, picks2mesh, picks2sphere, picks2ellipsoid, picks2plane,
  picks2surface and io.readers.coordinates use particle centres.

The shared helpers are in copick_utils.util.picks. Covered by
tests/test_pick_tools.py; every case fails on the previous code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@uermel @claude
chore: relock copick at the pushed core stack head (drop at release)
775528a 
uermel/panoptic-segmentations-v2 now points at dcbbd5e (untyped segmentation
queries select binary and multilabel); a5bf194 is no longer on a branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@uermel @claude
chore: require copick 2.0.0a3
e3c48c0 
copick 2.0.0a3 ships the v2.0 filament stack this builds on (copick#481, #482, #493, #495), so the build-against pin to the unreleased branch goes, and uv.lock resolves copick from PyPI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@uermel
uermel force-pushed the uermel/pick-identity-v2 branch from edbce53 to e3c48c0 Compare October 6, 2026 09:14
uermel added a commit that referenced this pull request Oct 6, 2026
@uermel @claude
fix: pick tools keep identity and place particles at their centre (#105)
8a944f0 
## Summary

Part 1 of 3 of filament support in copick-utils. Stack (`main`): #105 →
#106 → #107; `v2.0` twins #108 → #109 → #110. CLI docs:
copick/copick#496.

Requires copick 1.28.0, which ships the filament stack
(copick/copick#479, #480, #492, #494). The first commit built against
the unreleased stack and the last one sets the floor, so squashed, the
dependency change is `copick>=1.28.0`.

The pick tools now treat a pick as the particle it describes:

- `picksin`, `picksout` and `clippicks` test each particle's centre
(`location` plus the transform's translation) and write the surviving
points unchanged: transform, instance ID, score and order. An empty
result writes an empty pick set instead of failing and leaving stale
output behind.
- `clippicks --ref-seg` indexed a (Z, Y, X) volume with (X, Y, Z)
indices; it now indexes it correctly, also for non-cubic volumes.
- `picks2seg` and the pick-to-mesh converters (`picks2sphere`,
`picks2ellipsoid`, `picks2plane`, `picks2surface`, `picks2mesh`) place
each particle at its centre.
- Shared helpers in `util/picks.py`.

## Tests

`tests/test_pick_tools.py`: identity and order survive the filters, a
shifted particle is filtered and painted at its centre, a non-cubic
`--ref-seg`, an empty result. Each fails on `main`. The suite passes
locally (45 tests); `main` has no pytest CI.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
uermel added a commit that referenced this pull request Oct 6, 2026
@uermel @claude
feat: instance and panoptic segmentations in copick-utils (#106)
100d838 
## Summary

Part 2 of 3 of filament support in copick-utils. Stack (`main`): #105 →
#106 → #107; `v2.0` twins #108 → #109 → #110. CLI docs:
copick/copick#496.

copick now has instance segmentations (voxel value = instance ID) and
panoptic segmentations (a label channel plus an instance channel),
selected with `?instance=true` / `?panoptic=true` in a URI (copick
1.28.0: copick/copick#479, #480, #492, #494). This PR teaches
copick-utils to read and write them:

- **Framework.** Segmentation inputs and outputs take their type from
the URI; a command writes the type its `-o` names or refuses one it
cannot write. `filaments` joins picks, meshes and segmentations as a
worker input type. `io.readers.segmentation` and
`io.writers.segmentation` take the instance/panoptic flags. Untyped
selections stay binary and multilabel.
- **`separate-components`** with `-o ...?instance=true` writes one
instance segmentation (IDs 1..K by size) instead of K binary ones;
panoptic output from a multilabel or binary input. The per-component
loops are vectorised.
- **`filter-components`** drops whole instances by volume or by the new
`--min-skeleton-length`, keeping the surviving IDs; multilabel input
keeps its labels.
- **`seg-stats`** reports per-instance rows; `--skeleton` adds skeleton
length and label radius per component, for choosing thresholds.
- **`combine -o ...?panoptic=true`** builds a panoptic segmentation from
stuff labels plus per-object instance segmentations (`--instances`);
**`split`** takes a panoptic input apart again.
- `process/skeleton_graph.py`: a sparse 26-neighbour skeleton graph
(branches, junctions, spur pruning, junction merging), shared with the
filament tracer in part 3.

## Tests

`tests/test_instance_tools.py` (separate, filter and stats keep IDs; a
panoptic combine/split round trip; an untyped selection ignores a
same-key instance store) and `tests/test_skeleton_graph.py`. The suite
passes locally (55 tests).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
uermel added a commit that referenced this pull request Oct 6, 2026
@uermel @claude
feat: filament tracing (seg2fil, fil2picks, fil2seg) (#107)
314885e 
## Summary

Part 3 of 3 of filament support in copick-utils. Stack (`main`): #105 →
#106 → #107; `v2.0` twins #108 → #109 → #110. CLI docs:
copick/copick#496.

Filaments such as microtubules had no real path to picks: `fit-spline`
kept one path per volume, and `seg2picks` treats every object as a
sphere. Three new `copick convert` commands trace, sample and paint
filaments, sharing one set of IDs between copick's Filaments entity,
instance segmentations and picks (copick 1.28.0: copick/copick#479,
#480, #492, #494):

- **`seg2fil`** traces every filament in a binary, multilabel, panoptic
or instance segmentation. Lumen-sized holes are filled first
(`--fill-lumen`), so a tube labelled by its wall traces as one filament.
Components are skeletonized and split into branches; spurs are pruned,
nearby junctions merged, and branches continue through a junction where
they run most nearly straight (`--max-bend`), so crossing filaments stay
separate. Ends are extended to the label's edge. Short (`--min-length`),
blob-like (`--min-aspect`) and thin (`--min-radius`) pieces are
rejected. Each filament's smoothing spline is stored exactly as an
editable `bspline` curve; `--instances` also writes the instance
segmentation of the traced filaments. Tracing an instance segmentation
keeps its IDs.
- **`fil2picks`** samples each filament's curve (any curve kind, or one
derived from its points) at a required `--spacing`, with +Z along the
filament and rotation-minimising frames, grouped by filament ID and
ordered along it.
- **`fil2seg`** paints tubes around filaments into an instance
segmentation.

The old commands are kept and fixed, each pointing to the new ones:

- `process fit-spline` fits every filament instead of the longest
hop-count path, with no O(N³) fallback; spacing is exact (in voxels, now
documented); orientations come from the spline's derivative with
rotation-minimising frames; sharp fits are smoothed further instead of
deleting skeleton points; picks carry filament IDs. New `--label` and
`--filaments`. `--connectivity-radius` is hidden and has no effect.
- `process skeletonize`: new `--prune-length` prunes spurs
(`--remove-short-branches` only ever removed whole small pieces, as its
help now says); the `-o` name is honoured.

The tracer (`process/filament_tracing.py`, numpy/scipy) is public for
copick-pipeliner.

## Acceptance

On the 58 easymode microtubule segmentations of CZ cryoET Data Portal
dataset 10521, run through the CLI:

- 102 filaments (34.5 µm) give 4,255 picks at 82 Å, none on specks.
Consecutive picks are 79.9–82 Å apart, frames turn at most 29° between
neighbours, and the median distance to the label's medial axis is 5.6 Å
(octopi's watershed picks: 6.1 Å; `fit-spline` 1.8.1: 7.1 Å, covering
42% of the traced length).
- IDs agree across the instance segmentation, Filaments and picks;
re-tracing the instance segmentation returns the same IDs.
- Every stored curve is current; resampling at 41 Å needs no re-trace.
- `copick export picks --filament-columns on` writes valid RELION
filament columns, and re-importing them keeps IDs, order and poses
exactly.

## Tests

`tests/test_filaments.py` on synthetic tubes (straight, along Y, arc,
crossing, ring, thick end, hollow tube, speck, blob, sliver), plus the
converter-signature, run-names and legacy-flag pinning tests. The suite
passes locally (84 tests).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@uermel
uermel merged commit d764eb7 into v2.0 Oct 6, 2026
6 checks passed
uermel added a commit that referenced this pull request Oct 6, 2026
@uermel @claude
feat: instance and panoptic segmentations in copick-utils (v2.0) (#109)
1c1d8d9 
## Summary

Part 2 of 3 of filament support in copick-utils, `v2.0` line
(cherry-picked twin of the `main` PR). Stack (`v2.0`): #108 → #109 →
#110; `main` twins #105 → #106 → #107. CLI docs: copick/copick#496.

copick now has instance segmentations (voxel value = instance ID) and
panoptic segmentations (a label channel plus an instance channel),
selected with `?instance=true` / `?panoptic=true` in a URI
(copick/copick#481 → #482 → #493 → #495). This PR teaches copick-utils
to read and write them:

- **Framework.** Segmentation inputs and outputs take their type from
the URI; a command writes the type its `-o` names or refuses one it
cannot write. `filaments` joins picks, meshes and segmentations as a
worker input type. `io.readers.segmentation` and
`io.writers.segmentation` take the instance/panoptic flags. Untyped
selections stay binary and multilabel.
- **`separate-components`** with `-o ...?instance=true` writes one
instance segmentation (IDs 1..K by size) instead of K binary ones;
panoptic output from a multilabel or binary input. The per-component
loops are vectorised.
- **`filter-components`** drops whole instances by volume or by the new
`--min-skeleton-length`, keeping the surviving IDs; multilabel input
keeps its labels.
- **`seg-stats`** reports per-instance rows; `--skeleton` adds skeleton
length and label radius per component, for choosing thresholds.
- **`combine -o ...?panoptic=true`** builds a panoptic segmentation from
stuff labels plus per-object instance segmentations (`--instances`);
**`split`** takes a panoptic input apart again.
- `process/skeleton_graph.py`: a sparse 26-neighbour skeleton graph
(branches, junctions, spur pruning, junction merging), shared with the
filament tracer in part 3.

## Tests

`tests/test_instance_tools.py` (separate, filter and stats keep IDs; a
panoptic combine/split round trip; an untyped selection ignores a
same-key instance store) and `tests/test_skeleton_graph.py`. The suite
passes locally (78 tests).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
uermel added a commit that referenced this pull request Oct 6, 2026
@uermel @claude
feat: filament tracing (seg2fil, fil2picks, fil2seg) (v2.0) (#110)
76c3236 
## Summary

Part 3 of 3 of filament support in copick-utils, `v2.0` line
(cherry-picked twin of the `main` PR; `fil2seg` reads the tomogram shape
through `get_level_array`, and the package smoke test counts 35 entry
points). Stack (`v2.0`): #108 → #109 → #110; `main` twins #105 → #106 →
#107. CLI docs: copick/copick#496.

Filaments such as microtubules had no real path to picks: `fit-spline`
kept one path per volume, and `seg2picks` treats every object as a
sphere. Three new `copick convert` commands trace, sample and paint
filaments, sharing one set of IDs between copick's Filaments entity,
instance segmentations and picks (copick/copick#481 → #482 → #493 →
#495):

- **`seg2fil`** traces every filament in a binary, multilabel, panoptic
or instance segmentation. Lumen-sized holes are filled first
(`--fill-lumen`), so a tube labelled by its wall traces as one filament.
Components are skeletonized and split into branches; spurs are pruned,
nearby junctions merged, and branches continue through a junction where
they run most nearly straight (`--max-bend`), so crossing filaments stay
separate. Ends are extended to the label's edge. Short (`--min-length`),
blob-like (`--min-aspect`) and thin (`--min-radius`) pieces are
rejected. Each filament's smoothing spline is stored exactly as an
editable `bspline` curve; `--instances` also writes the instance
segmentation of the traced filaments. Tracing an instance segmentation
keeps its IDs.
- **`fil2picks`** samples each filament's curve (any curve kind, or one
derived from its points) at a required `--spacing`, with +Z along the
filament and rotation-minimising frames, grouped by filament ID and
ordered along it.
- **`fil2seg`** paints tubes around filaments into an instance
segmentation.

The old commands are kept and fixed, each pointing to the new ones:

- `process fit-spline` fits every filament instead of the longest
hop-count path, with no O(N³) fallback; spacing is exact (in voxels, now
documented); orientations come from the spline's derivative with
rotation-minimising frames; sharp fits are smoothed further instead of
deleting skeleton points; picks carry filament IDs. New `--label` and
`--filaments`. `--connectivity-radius` is hidden and has no effect.
- `process skeletonize`: new `--prune-length` prunes spurs
(`--remove-short-branches` only ever removed whole small pieces, as its
help now says); the `-o` name is honoured.

The tracer (`process/filament_tracing.py`, numpy/scipy) is public for
copick-pipeliner.

## Acceptance

On the 58 easymode microtubule segmentations of CZ cryoET Data Portal
dataset 10521, run through the CLI:

- 102 filaments (34.5 µm) give 4,255 picks at 82 Å, none on specks.
Consecutive picks are 79.9–82 Å apart, frames turn at most 29° between
neighbours, and the median distance to the label's medial axis is 5.6 Å
(octopi's watershed picks: 6.1 Å; `fit-spline` 1.8.1: 7.1 Å, covering
42% of the traced length).
- IDs agree across the instance segmentation, Filaments and picks;
re-tracing the instance segmentation returns the same IDs.
- Every stored curve is current; resampling at 41 Å needs no re-trace.
- `copick export picks --filament-columns on` writes valid RELION
filament columns, and re-importing them keeps IDs, order and poses
exactly.

## Tests

`tests/test_filaments.py` on synthetic tubes (straight, along Y, arc,
crossing, ring, thick end, hollow tube, speck, blob, sliver), plus the
converter-signature, run-names and legacy-flag pinning tests. The suite
passes locally (107 tests).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot mentioned this pull request Oct 6, 2026
Merged
uermel pushed a commit that referenced this pull request Oct 6, 2026
@github-actions
chore(v2.0): release copick-utils 2.0.0-alpha.3 (#115)
456b4e8 
🤖 I have created a release *beep* *boop*
---


##
[2.0.0-alpha.3](copick-utils-v2.0.0-alpha.2...copick-utils-v2.0.0-alpha.3)
(2026-10-06)


### ✨ Features

* filament tracing (seg2fil, fil2picks, fil2seg) (v2.0)
([#110](#110))
([76c3236](76c3236))
* instance and panoptic segmentations in copick-utils (v2.0)
([#109](#109))
([1c1d8d9](1c1d8d9))


### 🐞 Bug Fixes

* pick tools keep identity and place particles at their centre (v2.0)
([#108](#108))
([d764eb7](d764eb7))

---
Merging this PR will publish a copick-utils alpha prerelease to PyPI.

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@uermel