Skip to content

docs: add external-interop architecture for THEME-K (#125) #188

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
SoundMatt merged 1 commit into from
Aug 21, 2026
Merged

docs: add external-interop architecture for THEME-K (#125) #188

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
245 changes: 245 additions & 0 deletions docs/INTEROP-ARCHITECTURE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,245 @@
# External-Reference Interop Architecture — THEME-K

## Status

Design document. Precedes implementation of [THEME-K](https://github.com/SoundMatt/RELAY/issues/125)
and its instances INTEROP-01 through INTEROP-07 (issues #147–152). Mirrors
`RCP-ARCHITECTURE.md`'s role: a coordinating document for a change that spans
multiple repositories, written and verified against real CI configuration
before any repo's workflow is touched — not derived from the filing audit's
own text alone.

Every claim below was independently re-verified by reading the actual
`.github/workflows/*.yml` in each repo as it stands, not inferred from the
audit register. Several of the audit's own claims turned out to be wrong or
incomplete; those are called out explicitly rather than silently corrected,
per this project's standing practice of not trusting an audit claim that
hasn't been re-checked against current HEAD.

## The problem, precisely

`relay interop` (RELAY's own CLI, §20.2) is a **self-consistency check**: it
diffs a port's `convert` output against RELAY's own embedded golden vectors.
That is valuable — it catches "this port disagrees with RELAY's own reference
conversion" — but it structurally cannot catch "RELAY and every port sharing
the same misreading of the underlying wire protocol," because every input to
the check traces back to RELAY's own `spec/vectors/*.json`. Genuine
interoperability requires a peer that was **not** built against RELAY's
vectors: a real broker, a real kernel network device plus a third-party CLI,
or a vector set transcribed independently from the protocol's own external
standard.

## Current-state inventory (re-verified, not assumed)

| Family | External reference peer? | Probe-then-skip? | Exit-code discipline | Notes |
|---|---|---|---|---|
| RCP (go/c/cpp/rust) | No — RELAY-internal only; **c-RCP and cpp-RCP run no `relay interop` at all** | N/A | rust-RCP swallows the real exit code (see below) | Audit's "zero external interop" claim confirmed |
| LIN (go/cpp/rust) | No — RELAY-internal only | N/A | rust-LIN swallows the real exit code (see below) | Audit's "zero external interop" claim confirmed |
| MQTT (go/rust) | **Yes** — real `eclipse-mosquitto:2` service container + `mosquitto_pub`/`mosquitto_sub` | **Yes** — audit's claim of "no probe-then-skip" is **wrong**; both repos gate the live-broker job behind a probe step | Clean (real exit codes) | Strongest *functional* pattern (real, always-available broker), but not probe-free as claimed |
| DDS (go/cpp/rust) | Yes in principle, against CycloneDDS | Yes — **and for go-DDS/rust-DDS the probed Docker tag (`eclipse-cyclonedds/cyclonedds:latest`) was never published**, so the job plausibly never runs at all, not merely "sometimes skips" | Clean | cpp-DDS already fixed this by building the peer image locally from upstream source instead of pulling a nonexistent tag |
| CAN (go/cpp/rust) | Yes — real `vcan` kernel device + `can-utils` | Yes, but **honestly documented**: repos state in-line that GitHub-hosted runners lack the `vcan` kernel module and the job "routinely skips" | Clean | Most transparent skip messaging of any family; rust-CAN also correctly separates its loopback test (`can_two_process_interop`) from its real external test (`can_thirdparty_interop`) — a naming pattern worth generalizing |
| SOME/IP (go-SOMEIP) | No — no vsomeip or any external SOME/IP stack referenced anywhere in the repo | N/A | Clean | Audit confirmed |

**Mislabeling / exit-code violations found, by repo:**

- `rust-RCP` (`relay-interop` job) and `rust-LIN` (`relay-interop` job) both
run `output=$(relay interop ... 2>&1) || true` and then `grep` the captured
stdout for `EQUIVALENT`/`SKIP`/`ERROR` instead of checking the command's
real exit code. This silently tolerates both a genuinely broken run and any
future change to `relay interop`'s output formatting (§20.1 item 3 already
requires gating on the real result).
- No repo currently mislabels a same-process loopback test as "interop"
except by omission of a clearer name; rust-DDS's `rtps-interop`/
`shmem-interop` jobs are explicitly commented as distinct from external
interop already and don't need renaming. rust-CAN's
`can_two_process_interop` vs. `can_thirdparty_interop` split is the
ecosystem's best existing example of the naming discipline INTEROP-07
asks for — use it as the template, not an example needing a fix.
- No repo has an interop/conformance vector set independently derived from
the TC18 PDF or the LIN (ISO 17987) standard text. Every "golden vector"
check in every RCP/LIN repo traces back to `RELAY/spec/vectors/*.json`.

## Canonical conventions (apply ecosystem-wide, mechanical fixes first)

These four conventions resolve INTEROP-04, INTEROP-05, INTEROP-06, and
INTEROP-07 without requiring any new vector set. Each is independently
shippable as a small, low-risk PR per affected repo.

### 1. Naming discipline (INTEROP-07 + the mislabeling half of INTEROP-04)

- **`*-interop`** is reserved for a job/step that talks to a peer this
repository did not build: a real third-party binary, daemon, or CLI
(`mosquitto_pub`, `can-utils`'s `cangen`/`candump`, a real CycloneDDS
build, a future TC18/LIN reference peer).
- **`*-conformance`** (not "interop") is the correct label for any step
that only runs RELAY's own `relay interop`/`relay conform` CLI — that is
self-consistency against RELAY's embedded vectors, not third-party
interoperability. Rename existing `relay-interop` job/step names to
`relay-conformance` across every repo that currently calls them
"interop" while only invoking RELAY's own CLI.
- **`*-loopback`** or **`*-multiprocess`** is the correct label for a test
that runs two instances of this repo's own port against each other.
rust-CAN's `can_two_process_interop` naming is the reference example —
cite it directly when this convention lands in other repos.

### 2. Exit-code discipline (the `|| true` half of INTEROP-04)

`relay interop`/`relay conform` already return a real, meaningful exit code
(§11.3). No CI step may wrap that invocation in `|| true` and substitute a
`grep` of the captured stdout. Fix, concretely:

- **rust-RCP** `relay-interop` job: replace
`output=$(relay interop ... 2>&1) || true` + `grep` with a direct
invocation whose exit code gates the step (matching go-RCP's existing
clean pattern in the same family, which needs no change).
- **rust-LIN** `relay-interop` job: identical fix, matching go-LIN's and
cpp-LIN's already-clean pattern.

This is a pure regression-risk-free mechanical fix — the underlying check is
unchanged, only whether CI actually observes its result.

### 3. Probe-then-skip hardening (INTEROP-05) — with a prerequisite

A probe that can *genuinely* never succeed on the ecosystem's runners (the
`vcan` kernel module is documented as absent on GitHub-hosted runners; no
public `eclipse-cyclonedds/cyclonedds` image exists) is a different problem
from a probe that is simply *unreached* due to a bug. **Fix the bug before
hardening the gate**, or hardening will just turn a silent no-op into a
permanently red required check:

- **go-DDS, rust-DDS**: the `docker pull eclipse-cyclonedds/cyclonedds:latest`
step targets a tag that was never published — this is a real defect, not
environmental unavailability. Fix first by adopting cpp-DDS's already-
working pattern: build the peer image locally from upstream CycloneDDS
source (`docker compose ... build cyclone-peer`) instead of pulling a
nonexistent tag. Only once the probe can genuinely succeed does hardening
it make sense.
- **Once a probe's target is real** (CAN's `vcan`+`can-utils`, DDS's
locally-built peer, MQTT's service container), add a hard-failing check on
the canonical Linux runner specifically: if that probe still reports
unavailable there, fail the job instead of emitting `::notice::`/
`::warning::` and silently skipping. Other runners (e.g. a hosted runner
genuinely missing `vcan`) may still legitimately skip — the goal is that
*the* runner the ecosystem treats as canonical for gating a merge can no
longer report green without having executed the check.
- CAN's existing skip messaging (loud `::warning::` plus a
`$GITHUB_STEP_SUMMARY` banner, explicit in-line comment admitting the job
"routinely skips") is the right transparency floor even before hardening
— repos still using a quiet `::notice::` (go-mqtt, rust-MQTT, go-DDS,
rust-DDS) should upgrade to CAN's messaging as an interim step.

### 4. Service-container-first, where a real image exists (INTEROP-06)

MQTT's pattern —

```yaml
services:
mosquitto:
image: eclipse-mosquitto:2
ports:
- 1883:1883
```

— is the standard **wherever a suitable public image genuinely exists**,
because it guarantees the peer is present on every run rather than depending
on a runtime probe at all. This is not a drop-in replacement for every
family, though: CAN's peer is a kernel device (`vcan`), not a containerized
daemon, so a `services:` block doesn't apply; DDS has no official published
CycloneDDS image, so cpp-DDS's local-build-then-probe is the correct
adaptation of the same underlying principle (guarantee-first, not
runtime-luck-first) rather than a literal copy of MQTT's YAML. Apply the
*principle* (an unconditionally-present peer beats a probed one) family by
family, not the literal `services:` syntax where it doesn't fit.

## The RCP and LIN vector-harness gap (INTEROP-01, INTEROP-02) — scoped, not built here

This is deliberately **not** designed to completion in this document. Two
reasons:

1. Authoring genuinely independent TC18 or LIN-standard (ISO 17987) golden
vectors means reading the external specification text directly and
hand-deriving wire-level byte sequences, cross-checked against nothing
RELAY or any port already produces. That is a careful, slow, error-prone
exercise on its own — the opposite of something to rush inside a
coordinating architecture doc.
2. Unlike the mechanical conventions above, a wrong vector here doesn't just
miss a bug — it actively teaches every port the same wrong behavior a
second time, this time under the banner of "independently verified."
Getting the derivation process right matters more than getting it done
fast.

What this document does fix now: the **shape** of the harness, so that when
vector authoring happens (as its own, separately-scoped follow-on effort,
matching the phased "batch per protocol" pattern already used for this
session's `tc18_master_id` propagation work), every repo wires it in the
same way.

- **Vector location**: a new `spec/vectors/external/tc18/` (and, for LIN, a
parallel `spec/vectors/external/lin-iso17987/`) directory in RELAY,
distinct from `spec/vectors/*.json` (which remains RELAY's own reference
vectors, governed by §15.8/Requirement 16 and unaffected by this work).
Each external vector file records, at minimum, the TC18/ISO-17987
section and page/table reference it was transcribed from — the citation
*is* the independence proof; a vector with no citation back to the
external spec text is indistinguishable from one derived off an
implementation and provides none of the value this initiative exists to
add.
- **Harness shape**: a new `relay interop --external <vectors-dir>` mode (or
a sibling command if `interop`'s existing hub-spoke design — §20.1 item 3
— doesn't fit an externally-sourced fixture set cleanly; needs its own
design pass once vector authoring starts) that feeds a port's `convert`
output through the same equivalence check `relay interop` already uses,
but against `spec/vectors/external/` instead of `spec/vectors/`. This
reuses §20.1's existing machinery rather than inventing a second
conformance pipeline.
- **CI wiring**: once the vector set and harness command exist, each RCP/LIN
repo adds one job in the same shape as CAN's `can-interop` — a real,
named `tc18-interop`/`lin-iso17987-interop` job, required (no `|| true`,
no probe — the vector set is a checked-in file, not an environmental
dependency, so there is nothing to probe).
- **Sequencing**: RCP before LIN — RCP is the ecosystem's flagship protocol
per THEME-K's own framing and already has the deeper cross-repo tooling
investment (`docs/RCP-ARCHITECTURE.md`, the `tc18_master_id` propagation
work) to build on.

## What to ship first (recommended order)

Ordered by risk and independence — each is a standalone PR, no step depends
on a later one:

1. **INTEROP-04 exit-code fix** (rust-RCP, rust-LIN) — two small, safe PRs.
2. **INTEROP-07 / naming pass** — rename `relay-interop` jobs that only call
RELAY's own CLI to `relay-conformance`, ecosystem-wide.
3. **DDS Docker-image bug fix** (go-DDS, rust-DDS) — adopt cpp-DDS's
local-build pattern; this is a correctness fix independent of THEME-K's
broader framing and worth shipping even before the hardening step.
4. **INTEROP-05 hardening** — once (3) lands, add the canonical-runner
hard-fail for CAN, DDS, and MQTT's probes.
5. **INTEROP-06** — audit any family still on quiet `::notice::` skip
messaging and upgrade to CAN's transparent skip banner as an interim
step even before (4) lands everywhere.
6. **INTEROP-01/02 vector harness** — the large, separately-scoped,
multi-session effort described above. Not started by this document;
this document exists so that when it starts, it starts with an agreed
shape instead of four/three independent, incompatible designs.

## Verification standard for any change made against this document

Same standard as `RCP-ARCHITECTURE.md`: every claim re-checked against the
repo's actual current CI configuration before a PR is opened, not assumed
from this document or the originating audit. Any CI change lands with the
job actually observed to run (not just parse) in a real Actions run before
merge — a probe-hardening change in particular must be confirmed to fail
loudly when the peer is genuinely absent and pass cleanly when it's present,
not merely reviewed by reading the YAML.

## Provenance

Written 2026-08-21, following the THEME-K umbrella issue (#125) and its six
instances (INTEROP-01 through INTEROP-07, issues #147–152) filed from the
2026-07-29 ecosystem audit register. Every finding in the "Current-state
inventory" section above was independently re-verified by reading the real
`.github/workflows/*.yml` in each named repo, not taken from the audit
register's own text — this surfaced one incorrect audit claim (MQTT's
probe-then-skip) and one previously-undocumented real bug (go-DDS/rust-DDS's
CycloneDDS image tag was never published) that the audit itself missed.
Loading