Repository navigation
Conversation
f-dy
force-pushed
the
fix/subpixel-alignment-non-multiple-of-8
branch
from
September 23, 2026 15:18
bbbf546 to
3ad03f7
Compare
This was referenced Sep 23, 2026
Closed
The encoder halves three times and the decoder upsamples back through the sizes it recorded. Those are floor-halvings, so a dimension that does not halve exactly makes an F.interpolate ratio non-integral, and with align_corners=False the score map lands off the pixel grid. Only the two finest levels move a keypoint measurably, so detections break flip equivariance by up to half a pixel whenever a dimension is not a multiple of 4, following W % 4 on x and H % 4 on y independently. Widths 500 and 508 are unbiased despite not being multiples of 8. load_image() floors both dimensions to a multiple of 8, which is also a multiple of 4, so the documented path is unaffected; a caller that passes its own tensor to detect() is not. Pad the bottom-right up to a multiple of 4, then crop the score map back. Padding after the origin leaves the top-left alignment unchanged, so sizes that were already correct produce identical output. Padding to 8 instead would also make the coarsest halving exact; that was measured to move keypoints by 0.0025 px, so 4 is used as the smallest sufficient modulus.
f-dy
force-pushed
the
fix/subpixel-alignment-non-multiple-of-8
branch
from
September 23, 2026 15:34
3ad03f7 to
7dd9b9f
Compare
Author
|
Superseded by #19, which has the same change with a clean history. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #18
Keypoint coordinates are misaligned by up to half a pixel when an image dimension is not a multiple of 4.
Cause
forward_implupsamples the logits back through the sizes recorded by the encoder. Those are floor-halvings, so a dimension that does not halve exactly makes anF.interpolateratio non-integral, and withalign_corners=Falsethe score map lands off the pixel grid.Only the two finest levels move a keypoint measurably, so the condition is
% 4, not% 8:500 and 508 are the discriminating cases: not multiples of 8, but unbiased, because only the coarsest ratio is non-integral and that level is smoothed away.
load_imagefloors both dimensions with(scale * W) // 8 * 8. A multiple of 8 is also a multiple of 4, so that path satisfies the condition and never exposes this. A caller passing its own tensor todetectgets no such guarantee.Fix
Pad the bottom-right up to a multiple of 4, then crop the score map back. Padding after the origin leaves the top-left alignment unchanged, so sizes that were already correct produce identical output.
Padding to 8 instead would also make the coarsest halving exact. Measured against padding to 4, that moves keypoints by 0.0025 px and is bit-identical wherever both round to the same size, so 4 is used as the smallest sufficient modulus. Overhead is under 0.2% of a
detectcall (0.04 ms of ~23 ms on a T4).Measured
Trimmed-mean flip-equivariance residual, which must be 0.
to_pixel_coordsmaps indexitoi + 0.5on a[0, W]axis, so a flip mapsxtoW - xexactly. One image cropped to each size, so content is constant.x follows
W % 4and y followsH % 4independently: across 16 widths the y residual stays near -0.002, and across 16 heights the x residual stays near +0.010.More keypoints also survive the flip, since they no longer move off their true position (width 503: 929 to 1191 correspondences).
Tests
tests/flip_equivariance_test.pycovers one size per residue class on both axes. Without the fix 12 of 16 cases fail and the 4 that pass are exactly the% 4 == 0sizes, so it is not vacuous. With the fix all 16 pass, andtests/smoke_test.pyis unchanged at 1 passed.Found while measuring keypoint coordinate conventions through LoMa, which carries a copy of this detector and shows the same behaviour; davnords/LoMa#14 is the same patch there.
Draft because you may prefer the padding in
detectrather thanforward_impl.