Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions docs/known-limits.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,16 @@ Do not retry one without a new idea, and add a line here when you measure a new
202 unmatched instance observations still include real background pedestrians and people
pictured on signage. Do not animate every unmatched detection or call the single-track
result accepted. Its body projection also remains misaligned with the observed silhouette.
Experimental `--pose-crops` adds calibrated inference for unmatched masks, but has no GPU
quality result yet and remains off by default; see [person motion](person-motion.md#experimental-tracking-crop-recovery).
A third paid trial used experimental `--pose-crops` plus explicit source-reviewed exclusions
for 16 advertisement observations. Of 186 crop calls, 67 returned poses and only five passed
the unchanged mask-support test. All five scored 0.160–0.220, below the existing 0.30 threshold
for starting a track; no additional track survived. The main walker retained all 94 samples,
but background people remained missing in source overlays. The median crop joint-box height
was 0.406 observed body-heights, with a median center offset of 0.214 body-heights (image-space
diagnostics, not physical measurements). Do not treat support counts as surviving tracks,
lower thresholds to declare success, or repeat this crop recipe unchanged. All three ordinary
paid tracking executions are used. The option stays off by default; see
[person motion](person-motion.md#experimental-tracking-crop-recovery).
Evidence: `.context/evidence/city-recovery-20261120/`.

- The forest scale gate fails on distant background, not on the route. Replaying the run's own
Expand Down
29 changes: 26 additions & 3 deletions docs/person-motion.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,29 @@ change the stage retry budget, or distinguish pedestrians from people pictured o
statues. `detection-audit.json` records crop bounds, detections, recoveries and pose origins;
track summaries record call counts and motion observations preserve their origin.

The geometry and submission plumbing have CPU tests. Actual model recovery and visual
quality have not yet been established; do not enable this by default or treat recovered
detections as an accepted all-people reconstruction.
The geometry and submission plumbing have CPU tests. In the first city GPU trial, 186 crop
calls produced only five mask-supported poses; none met the unchanged track-start score.
No additional person track survived and source overlays still missed pedestrians. The reviewed
advertisement exclusions worked, but all-person reconstruction failed. Keep crop recovery off
by default; `recoveredCropPoses` counts supported detections, not surviving tracks.

## Reviewed depictions and statues

`--person-exclusions review.json` (or the tracking stage's `person_exclusions` parameter)
accepts explicit source-bound observations of a depiction or statue. It does not classify
objects automatically. Unreviewed observations remain eligible for pose recovery and tracking,
including small or stationary pedestrians. An exclusion must be visually reviewed with source
context; an agent-authored review must identify its author as an agent, not a human approval.

The `wander.person-exclusions/1` JSON includes `sourceSha256`, source `width` and `height`,
`reviewer` (`kind`, `name`), and `entries`. Each entry records `sourceIndex`, a source-pixel
`box` in xyxy order, `label` (`depiction` or `statue`), a specific `reason`, and an `evidence`
reference. Every reviewed box must uniquely match a detector box at IoU strictly above 0.95.
Stale source/dimensions, ambiguous matches and missing detections fail rather than silently
excluding something else. Entries outside the tracking schedule are rejected.

Excluded instances remain competing evidence during pose assignment, so a pose belonging to a
pictured person is not reassigned to its neighbor. They are recorded in `excludedInstances`
separately from unresolved `unmatchedInstances`; the exact review is retained with its SHA-256
and reviewer in the track receipt. The review content participates in the paid-attempt identity.
It does not change retry caps or establish all-person recall, identity or scene acceptance.
5 changes: 5 additions & 0 deletions orchestrator/steps.py
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,11 @@ def tunables(**knobs: dict[str, Any]) -> dict[str, Any]:
),
)
TRACK_PARAMETERS = tunables(
person_exclusions=knob(
"string",
"",
"path to source-bound reviewed depiction/statue exclusions; no automatic semantic approval",
),
pose_crops=knob(
"boolean",
False,
Expand Down
18 changes: 16 additions & 2 deletions scripts/run_clip.py
Original file line number Diff line number Diff line change
Expand Up @@ -1840,6 +1840,10 @@ def pose_guard(self):

def tracks(self):
"""Every person in the clip, linked into identity-stable tracks (one MultiHMR pass)."""
if getattr(self.a, "person_exclusions", None):
from worker.stages.person_exclusions import load_video_exclusions

load_video_exclusions(self.a.person_exclusions, self.clip, fps=self.a.fps)
self.paid_run(
"tracks",
[
Expand All @@ -1855,6 +1859,11 @@ def tracks(self):
"--det-thresh",
str(self.a.det_thresh),
*(["--pose-crops"] if getattr(self.a, "pose_crops", False) else []),
*(
["--person-exclusions", str(self.a.person_exclusions)]
if getattr(self.a, "person_exclusions", None)
else []
),
"--out",
str(self.ctx / "tracks"),
],
Expand Down Expand Up @@ -4359,6 +4368,9 @@ def main():
default=0.15,
help="MultiHMR detection threshold for the tracking stage",
)
ap.add_argument(
"--person-exclusions", default="", help="source-bound reviewed non-person instances"
)
ap.add_argument(
"--pose-crops",
action="store_true",
Expand Down Expand Up @@ -4446,8 +4458,10 @@ def main():
animation_options(execution_timeout=a.execution_timeout)
except ValueError as error:
ap.error(str(error))
if a.pose_crops and (not (a.people or a.all_people) or a.orchestrated):
ap.error("--pose-crops requires direct tracked runs; use wander step for stage parameters")
if (a.pose_crops or a.person_exclusions) and (not (a.people or a.all_people) or a.orchestrated):
ap.error(
"pose crops/exclusions require direct tracked runs; use wander step for stage parameters"
)
if a.lower_body_provenance is not None:
if not a.lower_body_provenance.strip():
ap.error("--lower-body-provenance must be a nonempty measured statement")
Expand Down
2 changes: 2 additions & 0 deletions scripts/stage_attempts.py
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,7 @@ def _read_authorization(path):
"worker/stages/track_people.py",
"worker/stages/pose_instances.py",
"worker/stages/pose_crops.py",
"worker/stages/person_exclusions.py",
),
"worker/modal_multiperson.py::animate": (
"worker/stages/worker_resources.py",
Expand Down Expand Up @@ -188,6 +189,7 @@ def _read_authorization(path):
}
TEXT_FLAGS = {"--experiment", "--mask-backend", "--alignment", "--depth-roi", "--refine-prompt"}
INPUT_FLAGS = {
"--person-exclusions",
"--image",
"--cameras",
"--masks-in",
Expand Down
Loading
Loading