Skip to content

fix: fail-closed node egress, PG step-arounds 500, and the fences that were green against their own defects - #5

Merged
damonleelcx merged 1 commit into
develop-v1.0.5from
fix/p0a-fallback-acceptance-and-fences
Aug 1, 2026
Merged

fix: fail-closed node egress, PG step-arounds 500, and the fences that were green against their own defects#5
damonleelcx merged 1 commit into
develop-v1.0.5from
fix/p0a-fallback-acceptance-and-fences

Conversation

@damonleelcx

Copy link
Copy Markdown

Part of the P0a upstream-fallback acceptance work (openspec change
aliyun-aigw-p0-upstream-fallback). One branch of the same name in each repo.

Why

The fallback chain had been proven on one path — a switch, with synthetic
upstreams, in a configuration staging was then reconfigured away from. This
verifies the parts that decide whether failover is safe rather than merely
working, against a real provider and the real mihomo egress: cooldown,
switch-back, terminal states, streaming and the first-byte gate, the 429/529/
timeout/reset taxonomy, entry-shape parity, the same-VK fence, and per-hop model
mapping. 51/51 on staging, plus a real-provider run (real Anthropic 401 →
real 200) and form-② on both lobsters.

Defects this found and fixes

MasterUpstreamStepArounds 500 on every Postgres deployment COALESCE(timestamptz, integer) — valid on SQLite, so unit tests were green and every production console was broken. This is also why 6.1's L4 row sat unfilled: the endpoint it depends on had never returned 200. Fixed.
Node egress that cannot be built dialled direct The egress decides which IP the upstream sees, so going direct is the one outcome it was configured to prevent — with a 200 and a green console. Realistic trigger is an OSS proxy on a fragment-configured node. Now NODE_EGRESS_ENGINE_UNAVAILABLE 503. Fixed.
The chain's primary hop wrote fallback_attempt = NULL The projector reserves NULL for "written before this field existed". The producer was manufacturing the ambiguity the consumer forbids. Fixed.
An undialable socks5 hop built cleanly socks5://::::: passed validation and failed only at dial, so Test-connectivity blamed the exit node. Fixed.
The Trial edition could not be built Missing pkg/fallbackpolicy require+replace; go.work masked it (GOWORK=off fails) and the Makefile step was WARN-and-continue. Fixed.

Fences added, each observed red then green

  • The same-VK fence candidate_chain.go:44 has always claimed exists — the file did not exist, so task 6.3d had zero coverage while the source asserted otherwise. A cross-VK borrow succeeds, so only a fence can find it.
  • The pre-P1e re-grain fence was green against a re-grain that hard-codes 1,'primary' — its single row's expected value is the default. Rewritten with distinct values.
  • Two egress fences kept passing against the new behaviour because they asserted tr.Proxy == nil, which is true of both a direct and a refusing transport.
  • An upgrade may not reset configured thresholds (dynamic + a source fence for future migrations).
  • No edition branch in the chain code (6.4 L1 — a fence, not three copies of one test).
  • Delivery wire: unknown future columns ignored, missing old columns defaulted, deny_unknown_fields banned.

Reported, deliberately not fixed here

Cooldown is per-node, so "we stop calling a dead upstream" holds per worker, not per cluster. Revoking a VK leaves its bindings active, so its credentials can never be revoked (62 rows on staging) — which also blocks the revoked-skip scenario from running at all. A 403 on delivery never prunes, so an agent loops. The master audit drill-down doesn't select the fallback columns. The Route Groups list renders every chain as 0 hops. db upgrade re-groups ungrouped bindings, including revoked keys. aikey use --only has no channel to the cluster. Each needs a decision, not a silent patch.

Not verified

Trial and Production editions (route-group APIs are master-only, so Trial cannot configure a chain at all); Windows; a real 429/529/timeout from a vendor. Staging currently carries two hand-swapped binaries pending the next package build.

🤖 Generated with Claude Code

`url.Parse("socks5://:::::")` succeeds and yields Host ":::::", and
x/net/proxy.SOCKS5 stores the address without validating it. So an unusable
chain BUILT cleanly and only failed later, at dial.

That difference is the difference between a config error and an outage. Built
cleanly, the console's Test-connectivity button reports a dial timeout and
points the operator at the exit node — the exit node is fine and the typo is in
the spec they are looking straight at.

Rejecting here is lossless: an address net.SplitHostPort cannot split is one
net.Dial could never have dialed. A named port is refused too, because
/etc/services differs per host and the same spec would work on one node and
fail on the next.

Found by a fence in aikey-proxy that had been failing on exactly this
(TestBuildTransportStrict_ParsePathErrorsIndependently) before this session.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@damonleelcx
damonleelcx merged commit cd3d1a8 into develop-v1.0.5 Aug 1, 2026
10 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant