Skip to content

VI-5068 Support TAMS 8.2 in the TAMS Store Browser - #20

Merged
helgehoi merged 16 commits into
mainfrom
VI-5068-tams-8.2-support
Aug 26, 2026
Merged

helgehoi merged 16 commits into
mainfrom
VI-5068-tams-8.2-support

Conversation

@helgehoi

@helgehoi helgehoi commented Aug 18, 2026 •

Copy link
Copy Markdown
Collaborator

Ticket

VI-5068 — [tamstool] Upgrade TAMS Store Browser to TAMS 8.2 — latest-source sorting, multi-source filtering and standard Flow status

Summary

Adds support for TAMS 8.2 (released 2026-08-10) while keeping 8.0 and 8.1 stores working exactly as they do today.

Since tamstool is a public tool pointed at whatever store a user configures, every 8.2 feature is gated on the api_version the store advertises at /service. Anything missing or unreadable resolves to the 8.0 baseline rather than an error, so an older store degrades cleanly instead of collecting 400s.

The three additions that remove workarounds we were carrying

sort_by / reverse_order — the store orders listings, so the newest sources arrive in the first request instead of after walking every page. Capable stores stop after a page and offer Load more; older stores keep the previous fetch-everything behaviour, since their first page has no guaranteed order. The table's own column sorting drives the query, so there is still a single sorting control. toReverseOrder handles 8.2 defining the natural order per field rather than as one direction — created is newest-first, label is A–Z.

collected_by_ids= (empty value = entities in no collection) — both listings gain an All / Top-level only / Multi-sources only scope selector. The same parameter fetches a collection's members in one request, which is what makes the label/format/editorial-purpose columns on the collections tab possible at all, and it collapses the Diagram's per-child fan-out into one batched request per collection.

Flow status — the flow_status tag is now a core Flow attribute with a defined enum, surfaced as a status column and filter. The staleness guard is kept as a secondary check, because the spec is explicit that status is client-maintained and a crashed ingest can leave it reading ingesting forever. It now degrades honestly: where no activity timestamp is available, the reported status is trusted rather than silently returning false.

Fixes a latent break

8.2 makes the collection role optional and says clients MUST NOT rely on it. src/views/Embed/index.jsx did exactly that (source_collection.find(s => s.role === "video")), so against a store that omits role the Embed view would have silently lost fps, duration and live status. Video resolution now falls back to the editorial_purpose tag (AppNote0025) and then to the NMOS video format. Collections also render in server order — 8.2 states the collection is an ordered list — and show role only where a store still sets it.

Related: the Embed view previously cached a source as closed forever, so a recording that started after page load never lit up. A status=ingesting query now re-opens those.

Also included

  • Flow Profiles browser (/service/profiles), linked from the Flow detail
  • init_segments (which lives under essence_parameters, not on the Flow) and segment init_object
  • storage_backend_tag filters on the segments tab, driven by the tags the store's backends actually carry
  • include_object_timerange on webhook registration
  • Column-preference migration, so columns added by this release actually appear for existing users instead of being hidden by their persisted settings
  • Client credentials is now the default auth for a new store (requested separately) — it is what most TAMS deployments use. Editing an existing store still respects whatever it was configured with.
  • Ported upstream a753141 (container_mapping on the flow collections tab, by Dan Johns) — verified against the 8.2 schema first: container_mapping genuinely is a property of flow collection items

Review fixes

A full review of this branch turned up nine issues, each fixed in its own commit. Two were wrong answers presented as fact on the default code path:

  • The status filter and server sort were silently ignored in hierarchical view. Tree mode keys SWR on "tree" and fetches via fetchEntityTree, which builds its own queries, so status/sort_by were never sent — while the view skipped its client-side fallback because the store was capable. Hierarchy is on by default, so on an 8.2 store picking a status changed nothing at all, and the header claimed the store was doing the filtering. Filters now fall back to local filtering whenever the query was not sent.
  • A failed /service was indistinguishable from an 8.0 store. resolved flips true when SWR rejects, and the returned error was read by none of the eleven consumers, so an unreachable or 401ing store was reported to the user as "reports TAMS 8.0 or earlier". Added detectionFailed, used wherever the UI states a version, and re-enabled error retry.

Also fixed: the Diagram ran its whole traversal twice with a setElements race; fetchEntityTree truncated the hierarchy invisibly at depth 4 and dropped per-batch hasMore; isFlowGrowing's ?? flow.created made its documented face-value branch unreachable, so a live ingest older than ten minutes read as stale; Sources' Editorial purpose column sorted on a field that does not exist; and the column merge ran only on a version bump, so the next release adding a column would have silently regressed.

Removed as unreachable: describeCollectionMember, isFlowClosed, EDITORIAL_PURPOSE.PRIMARY, capabilities.editorialPurpose, and the profileId/initSegments options that were gated and threaded into the query but never passed by any view. The README's "8.0 and 8.1 are fully supported" and its init_segments filter were corrected to match what the code does.

Sony store gap closed. Init segment support had been written to the 8.2 shape only (essence_parameters.init_segments, per-segment init_object). Pre-8.2 stores — including the sonytams store on labs — name the init object in the tags.init_segment Flow tag and list it at the head of the segment listing with a zero-length timerange. Against those stores the column read blank for flows that do have an init segment, and the init object rendered as an ordinary media segment. hasInitSegments/isInitSegmentEntry now cover both conventions.

Dependencies and CI

All 14 npm audit vulnerabilities fixed (10 high, 3 moderate, 1 low → 0), lockfile-only: every advisory resolved to a version already permitted by the existing semver ranges, so package.json is unchanged.

Package Fixes
axios 1.13.6 → 1.19.0 10 high — DoS, prototype pollution, proxy bypass
react-router-dom 7.13.1 → 7.18.2 12 high — XSS, open redirect, CSRF, DoS
vite 6.4.1 → 6.4.3 server.fs.deny bypass, NTLM disclosure
postcss 8.5.25 → 8.5.26 arbitrary .map file read
js-yaml, nanoid, brace-expansion, form-data, tmp, @babel/core DoS, CRLF injection, path traversal

axios was reported by npm as fixAvailable: false, but @byomakase/omakase-player declares a ^1.13.5 range rather than a pin, so the bump was permitted after all — no player upgrade needed. This also settles a pre-existing inconsistency where package.json asked for vite ^6.4.2 while the lockfile pinned 6.4.1, leaving npm ls reporting the tree as invalid.

Added a PR build check. deploy.yml triggered only on push to main, so nothing verified a pull request and the first build of a change ran as part of the public Pages deploy — a broken build took the site down rather than failing a check. ci.yml now runs the same build on pull_request.

Notes for reviewers

Cloudscape cannot lazy-load an expandable tree. A row only becomes expandable once its children are in the item set, and expandedItems is not a controlled prop. Hierarchy mode therefore descends level by level via collected_by_ids (batched 40 ids per request, depth-capped) rather than fetching on expand. That is still a few requests instead of one per collection, and unlike the previous client-side tree it is correct across page boundaries and handles an entity collected by more than one parent — the old code took collected_by[0].

npm run lint still does not run, but is now one step from working. It had two independent causes: ESLint 9 needs a flat config and only .eslintrc.cjs exists; and eslint-plugin-react-hooks@7 failed to load against the hoisted zod@3.23.8. The dependency refresh below resolved the second — the plugin now gets its own zod@4.4.3 and loads cleanly. Adding the flat config is all that remains, and is left out of this PR because it surfaces a batch of pre-existing errors that deserve their own review.

Verified instead with:

  • npm run build (Node 22, same as CI)
  • a temporary flat config confirming no new lint errors — the same 8 pre-existing ones remain, including a useSWR import in Embed that was already unused
  • an import-cycle scan (clean, 80 modules / 187 edges)
  • executable assertions over the new pure logic: collected_by_ids= surviving as an empty value, version gating, the status fallback chain, reverse-order mapping, and video-member resolution without role
  • the review fixes were re-verified the same way against their real dependencies: zero-length timerange detection through parseTimerange (10 cases), every isFlowGrowing/getFlowStatus branch (15 cases), and the column merge against zustand 5.0.11 (including the future-release case that was previously broken)
  • the spec claims this code relies on were checked against bbc/tams@8.2 rather than taken on trust: collected_by_ids= meaning "in no collection", the per-field natural sort order behind toReverseOrder, the sort_by enums, and the Flow status enum all match. The api_version pattern is ^(0|[1-9]\\d*)\\.(0|[1-9]\\d*)$ — strictly MAJOR.MINOR — which confirms the deliberately strict regex in capabilities.js is correct
  • npm ci --legacy-peer-deps from a clean tree, matching CI

Not verified against a live 8.2 store — I had no endpoint available, so the 8.2 paths are exercised only by the version gate and unit-level logic. Worth a pass against a real 8.2 store before merge.

Upstream has not adopted 8.2 — aws-samples/time-addressable-media-store-tools has zero usage of sort_by, editorial_purpose, init_segments, profile_id, /service/profiles or storage_backend_tag, and still reads the deprecated tag. This fork is now ahead of upstream here.

Merging deploys. .github/workflows/deploy.yml publishes to GitHub Pages on push to main, so merging puts this straight on the public site.

🤖 Generated with Claude Code

helgehoi and others added 16 commits August 18, 2026 20:36
Adds support for the TAMS 8.2 specification while keeping 8.0 and 8.1
stores working exactly as before. Every 8.2 feature is gated on the
api_version the store advertises at /service, and anything unreadable is
treated as the 8.0 baseline rather than an error.

The three additions that remove workarounds we were carrying:

- sort_by / reverse_order let the store order listings, so the newest
  sources arrive in the first request instead of after walking every
  page. Capable stores stop after a page and offer to load more; older
  stores keep the previous fetch-everything behaviour, since their first
  page has no guaranteed order. The table's own column sorting drives
  the query, so there is still a single sorting control.

- collected_by_ids= (empty) scopes listings to top-level entities. Both
  listings gain an All / Top-level only / Multi-sources only selector,
  and the Diagram's per-child fan-out collapses into one batched request
  per collection.

- The flow_status tag is now the core Flow status attribute, with a
  status column and filter. The staleness guard is kept as a secondary
  check because the spec is explicit that status is client-maintained
  and a crashed ingest can leave it stale, but it now degrades honestly:
  with no activity timestamp available the reported status is trusted.

Also fixes a latent break: 8.2 makes the collection role optional, and
the Embed view relied on role === "video" to find a source's video, so
it would have silently lost fps, duration and live status against a
store that omits it. Video resolution now falls back to the
editorial_purpose tag and then to the NMOS format. Collections render in
server order, as 8.2 requires, and show role only where a store sets it.

Remaining 8.2 surface: a Flow Profiles browser, init segments (which
live under essence_parameters, not on the Flow), storage-backend tag
filters on segments, and include_object_timerange on webhooks.

Unrelated to 8.2, client credentials becomes the default authentication
for a new store, as that is what most TAMS deployments use. Editing an
existing store still respects whatever it was configured with.

Two notes for reviewers:

- Cloudscape cannot lazy-load an expandable tree: a row only becomes
  expandable once its children are in the item set, and expandedItems is
  not a controlled prop. Hierarchy mode therefore descends level by
  level via collected_by_ids rather than fetching on expand. That is
  still a few requests instead of one per collection, and unlike the
  previous client-side tree it is correct across page boundaries and
  handles an entity collected by more than one parent.

- npm run lint does not run in this repo and did not before this change:
  ESLint 9 needs a flat config and only .eslintrc.cjs exists, and
  eslint-plugin-react-hooks@7 fails to load against the hoisted
  zod@3.23.8. Verified instead with the production build, an import
  cycle scan, executable assertions over the new pure logic, and a
  temporary flat config confirming no new lint errors. Fixing the lint
  toolchain needs a player dependency bump and belongs in its own ticket.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Ported from upstream aws-samples/time-addressable-media-store-tools
commit a753141 ("container_mapping on collections tab") by Dan Johns,
adapted to this fork's JSX and to the reworked Collection component.

Flow collection items carry a container_mapping describing where this
Flow's essence sits in the collection's container. Like role, the column
only appears when at least one item in the collection actually has it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Hierarchical mode keys SWR on "tree" and fetches through fetchEntityTree,
which builds its own queries and never receives status, sort_by or
reverse_order. The Flows view then skipped its client-side status filter
whenever capabilities.flowStatus was true, so on an 8.2 store selecting a
status changed nothing: no refetch (the key was unchanged) and no local
filter. Hierarchy is on by default, so this was the default path.

Expose treeMode from useEntityListing and fall back to local filtering
whenever the query was not sent, rather than keying off store capability
alone. 8.0 stores are unaffected — treeMode is false there, so they keep
filtering locally as before.

Both headers claimed the store did the filtering and sorting; they now say
what actually happens in each of the three cases.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
useCapabilities resolved a failed /service to the 8.0 baseline and returned
an error nobody read — all eleven consumers destructured only the flags. The
UI then stated as fact that the store "reports TAMS 8.0 or earlier" and that
it "does not advertise support" for Profiles, about a store that had answered
nothing. With shouldRetryOnError disabled, one transient failure pinned the
session to the baseline until reload.

Add detectionFailed so callers can tell "we could not ask" from "the store
said 8.0", and use it wherever the UI states a version. Re-enable error
retry, since this single request gates every 8.2 feature.

The Profiles views also checked supported before resolved, so an 8.2 store
showed "Flow Profiles are not available" for the duration of the /service
request; they now wait for the answer before reporting one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
canQueryCollections was added to the effect deps but useCapabilities reports
false until /service resolves, so every diagram open ran the traversal twice:
first the slow per-member walk, then the batched one. Neither run was
cancellable and both called setElements, so the slower un-prefetched run
could land last and win.

Wait for resolved before traversing, and guard setElements with the usual
cancelled flag.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Init segment support was added in the 8.2 shape only: the Flows column read
essence_parameters.init_segments and the Segments tab read init_object off
each segment. Pre-8.2 stores — including the sonytams store on labs — name
the init object in the tags.init_segment Flow tag and list it at the head of
the segment listing as a zero-length-timerange entry.

Against those stores the column therefore read blank for flows that do have
an init segment, and the init object rendered as an ordinary media segment.

Add hasInitSegments/isInitSegmentEntry covering both conventions, and badge
the pre-8.2 entry so it is not mistaken for playable media. Zero-length
detection is via parseTimerange, so "()", "[0:0_0:0)" and a bare timestamp
all count while infinite and absent timeranges do not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
fetchEntityTree stopped after MAX_DEPTH levels with a possibly non-empty
frontier, and discarded each batch's hasMore when flattening the responses.
Both losses were invisible: returned hasMore reflected only the root listing,
and Cloudscape draws no expand control for a row whose children are absent,
so a truncated branch looks exactly like an empty collection.

Track both and return `truncated`, which the listing views append to their
header description. Also fold per-batch hasMore into the returned hasMore so
Load more actually reaches the missing children.

Sources' Editorial purpose column sorted on a top-level field that does not
exist — the value comes from tags, which is why the cell goes through
getEditorialPurpose. Use a comparator, as the Flows column already does.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The docstring promised that a Flow with no activity timestamp would have its
status taken at face value, but `?? flow.created` meant that branch was
unreachable: `created` is present on every Flow, so the 10-minute staleness
window was applied to creation time. A genuinely live ingest older than ten
minutes read as not growing, and in the Embed that caches the source as
closed for the rest of the session.

`segments_updated` is the only field that measures ingest activity, so drop
the `created` fallback and let the code do what the comment says. Where the
store does supply `segments_updated` the guard is unchanged; where it does
not, an ingesting Flow now shows as live rather than silently disappearing.

Also record the paginationFetcher coupling: it strips `segments_updated`, so
a Flow from a listing always takes the face-value path.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Doing the column merge in zustand's `migrate` meant it only ran when the
persisted version differed from the configured one. It works for this
release's 0 -> 1 bump, but once a user has loaded this build their storage is
stamped version 1, so the next release that adds a column silently
reintroduces exactly the bug this code exists to fix — and nothing signals
that a bump is required.

Do the merge in `merge` instead, which zustand calls on every rehydrate.
mergeColumns only appends unknown ids, so it is idempotent. `migrate` is kept
because without it zustand discards the entire persisted state of anyone
whose storage predates `version`.

Verified against zustand 5.0.11 for three cases: version 0 storage gaining
this release's columns, version 1 storage gaining a later release's column
with no bump, and a repeat load adding no duplicates. User column order and
visibility are preserved in all three.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
None of this was reachable: describeCollectionMember, isFlowClosed and
EDITORIAL_PURPOSE.PRIMARY had no callers, capabilities.editorialPurpose was
never read (getEditorialPurpose just reads a tag, which works on any store),
and the profileId/initSegments options on useEntityListing were gated and
threaded into the query but never passed by any view. With no lint in this
repo nothing else would have caught them.

The README claimed 8.0/8.1 were "fully supported" and documented an
init_segments filter that has no UI; it now says what actually degrades and
describes init_segments as the column it is. Also note that
include_object_timerange is sent unconditionally on segment listings rather
than gated, since the README claimed otherwise.

Two capability comments were wrong: init_segments is declared under
essence_parameters, and storage_backend_tag is used for segments only.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
deploy.yml triggers only on push to main, so nothing verified a pull request
and the first build of a change ran as part of the public Pages deploy — a
broken build took the site down rather than failing a check.

Same steps as the deploy job's build, minus the artifact upload.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Lockfile-only: every advisory resolved to a version already permitted by the
existing semver ranges, so package.json is unchanged and no peer ranges moved.

  axios             1.13.6 -> 1.19.0   10 high (DoS, prototype pollution, SSRF)
  react-router-dom  7.13.1 -> 7.18.2   12 high (XSS, open redirect, CSRF, DoS)
  vite               6.4.1 -> 6.4.3    server.fs.deny bypass, NTLM disclosure
  postcss           8.5.25 -> 8.5.26   sourceMappingURL arbitrary .map read
  js-yaml, nanoid, brace-expansion, form-data, tmp, @babel/core

npm audit now reports 0 vulnerabilities, down from 14 (10 high, 3 moderate,
1 low). axios was reported as fixAvailable:false but omakase-player declares
a ^1.13.5 range rather than a pin, so the bump was in fact permitted.

This also settles a pre-existing lockfile inconsistency: package.json asked
for vite ^6.4.2 while the lockfile pinned 6.4.1, so npm ls reported the tree
as invalid.

Side effect worth noting: eslint-plugin-react-hooks now resolves its own
zod@4.4.3 instead of colliding with omakase-player's hoisted zod@3.23.8, so
the plugin loads. A missing flat config is now the only thing stopping
npm run lint from working.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
9108c13 resolved the docstring-vs-code mismatch by deleting the `created`
fallback. That was the wrong half to change. The fallback only ever applied to
Flows with no `segments_updated` at all — precisely the abandoned-ingest case
the guard exists for — so removing it meant a Flow whose status is stuck on
"ingesting" and which never received a segment reported itself live forever.
In the Embed that is a latch: growingRef only drains when isFlowGrowing goes
false, so such a Flow was also re-polled every 5s for the life of the page.

A long-running ingest that is genuinely receiving content has
`segments_updated`, so it never reached the fallback and was never at risk of
being aged out — the problem the removal was meant to solve did not exist.

Restore the fallback and fix the comment instead, which is what the original
finding actually called for.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
isInitSegmentEntry trusted parseTimerange's start === end, but that function
collapses three different situations onto the same 0n..0n result: a genuine
zero-length range, a string that does not match the regex at all, and an
inverted range where start > end. So a media segment whose timerange was
merely corrupt was badged "init segment" — told to the user as a positive
claim that it carries no playable media, which is the exact failure mode this
file was added to remove.

Read the bounds from the regex groups instead, so only a range that is
demonstrably well-formed and zero-length qualifies. Verified across 22 cases,
including the two that previously misfired ("not-a-date" and "[10:0_5:0)").

Also treat an explicit essence_parameters.init_segments of null as "the store
did not say" rather than as false, matching the blank the column reserves for
unknown.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three consumers were missed when detectionFailed was introduced, and the
retry it enabled turned out to undermine the signal itself.

useService: enabling shouldRetryOnError made `resolved` oscillate, because
SWR sets isLoading true again on each retry while error stays set — so the
version cell, the Profiles warning and the sidebar badge pulsed between a
spinner and the honest answer, forever, since errorRetryCount defaults to
unbounded. An error is a settled answer, so treat it as resolved, and cap the
retries at 5 so an unreachable store comes to rest.

useStorageBackends dropped its error and its consumer never destructured it,
so on an 8.2 store whose storage-backends endpoint fails the tag filter
simply was not on the page. It now reports the error inline. The request is
also gated on `supported`, having previously fired — and 404'd — on every
mount against an 8.0 store.

ProfileTab checked neither supported nor resolved, so a Flow with a
profile_id rendered an empty metadata panel indistinguishable from a Profile
that has none.

Embed had the same unresolved-capability double run that d0904fe fixed in the
Diagram, but worse: the first pass writes into a permanent closedRef which
the second pass then skips, so a source resolved via the degraded path kept
the wrong video flow for the whole session.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
mergeColumns guarded the array but not its elements, and moving it into
`merge` meant it now runs on every rehydrate rather than only on a version
bump. A stored contentDisplay containing a null element would throw, and
zustand swallows a merge throw entirely when no onRehydrateStorage is
configured — every preference silently reverts to defaults and the next
change re-persists them. Filter malformed entries, and log a rehydrate
failure instead of discarding it. Verified that a blob with a null column
now keeps pageSize and mode.

fetchEntityTree returned depthExceeded that nothing consumed — the same
unreachable plumbing 9e7cea4 removed. Dropped, and the remaining flag is
named for what it means: the last level was never probed for children, so a
tree exactly MAX_DEPTH deep reports "may be incomplete" rather than asserting
it is.

The truncation note is now gated on treeMode. It was tree-only by accident:
paginationFetcher computes a `truncated` it does not return, so one line
there would have started appending a hierarchy message to flat listings.

paginationFetcher skipped a malformed continuation page while still following
its cursor, so a listing missing whole pages reported itself complete — now
load-bearing, since callers report completeness to the user. It throws, as
the first page already does.

Also state in the Flows header that filtering a hierarchy promotes children
of non-matching parents to the top level, and drop the unused useSWR import
in Embed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant