fix(ingress): wildcard gets its own Ingress + Traefik router.priority (root cause now measured) - #516
Open
izzywdev wants to merge 1 commit into
Open
fix(ingress): wildcard gets its own Ingress + Traefik router.priority (root cause now measured)#516izzywdev wants to merge 1 commit into
izzywdev wants to merge 1 commit into
Conversation
Makes the tenant-subdomain wildcard safe to re-enable. It stays DISABLED in prod — this ships the mechanism, not the rollout. #431 shipped the wildcard as a second rule on the main Ingress and took FuzePlan down: plan.fuzefront.com served the FuzeFront shell. #437 reverted it. The root cause was recorded then as INFERRED. It is now measured. MEASURED on rancher/mirrored-library-traefik:3.6.13 (the exact prod image), file provider, two routers, neither declaring a priority: Host: plan.fuzefront.com -> wildcard-fuzefront@file -> :9002 WRONG The exact-host router lost to the wildcard. Traefik orders routers by RULE LENGTH, and the generated HostRegexp is longer than Host(`plan.fuzefront.com`). With `priority: 1` on the wildcard router only: Host: plan.fuzefront.com -> exact-plan@file -> :9001 correct Host: corpabc.fuzefront.com -> wildcard-fuzefront@file -> :9002 correct So the fix is to de-prioritise the wildcard. `router.priority` is a PER-INGRESS-OBJECT annotation, so the wildcard cannot be de-prioritised while it shares an object with the canonical host — hence the split into `fuzefront-tenant-wildcard`. The YAML anchor is replaced by a direct include of the same helper: an anchor cannot be referenced across documents, and the guarantee it existed for (the two fan-outs cannot drift) is preserved because there is still exactly one definition. Asserted in the render, not just claimed. Renders: prod (wildcard off) -> tenant-wildcard NOT rendered; main = 1 rule, 8 paths local (nginx) -> tenant-wildcard rendered, no priority annot (inert on nginx, which follows the k8s exact-beats-wildcard rule) prod DRY-RUN (on) -> router.priority=1, no tls: block, fan-out identical to canonical (8 paths), main still a single rule helm lint -> 0 failed Prod re-enablement remains a deploy-window decision: the wildcard CNAME already exists at Cloudflare, so flipping wildcardHost puts every *.fuzefront.com host live immediately. Verify plan.fuzefront.com straight after the sync. Refs #431, #437, FFRNT-91. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Automated code review (gate-code-review)Credit balance is too low Report-only — this check never blocks merge. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Makes the tenant-subdomain wildcard safe to re-enable. It stays disabled in prod — this ships the mechanism, not the rollout.
The root cause is no longer inferred
#431 shipped the wildcard as a second rule on the main Ingress and took FuzePlan down. #437 reverted it, and recorded the cause as inferred — the prod Traefik API was unreachable, so the router priorities were never read.
I reproduced it in isolation on
rancher/mirrored-library-traefik:3.6.13, the exact prod image, using the file provider with two routers and no declared priority:The exact-host router lost to the wildcard. Traefik orders routers by rule length, and the generated
HostRegexpis longer thanHost(`plan.fuzefront.com`).With
priority: 1on the wildcard router only:Why a separate Ingress object
traefik.ingress.kubernetes.io/router.priorityis a per-Ingress-object annotation. The wildcard cannot be de-prioritised while it shares an object with the canonical host — so it moves tofuzefront-tenant-wildcard. The template says so inline, with the measurement, so nobody merges it back.On ingress-nginx (local) the annotation is inert: nginx follows the Kubernetes spec and prefers exact hosts on its own.
The YAML anchor had to go
An anchor cannot be referenced across YAML documents, so
*fanoutcould not survive the split. Both call sites nowincludethe same helper. The guarantee the anchor existed for — the two fan-outs cannot drift — is preserved, because there is still exactly one definition. The render asserts it rather than trusting it.Renders
tenant-wildcardnot rendered; main = 1 rule, 8 pathstenant-wildcardrendered, no priority annotation (inert on nginx)router.priority=1, notls:block, fan-out identical to canonical (8 paths), main still a single rulehelm lintProd re-enablement is still a deploy-window decision
Deliberately not flipped here. The wildcard CNAME already exists at Cloudflare, so setting
wildcardHostputs every*.fuzefront.comhost live immediately — there is no DNS gate to stage behind. That assumption is what made #431 an outage rather than a no-op.When you do enable it, verify
plan.fuzefront.comstraight after the sync, not the rendered YAML.helm template | kubeconform -strictand a parsed fan-out-identity check both passed on #431 and neither could see cross-namespace router precedence.Refs #431, #437, FFRNT-91.
🤖 Generated with Claude Code