[WIP] feat: implement Gateway API routing provider + standalone notebooks - #1301
Draft
aojea wants to merge 10 commits into
Draft
[WIP] feat: implement Gateway API routing provider + standalone notebooks#1301aojea wants to merge 10 commits into
aojea wants to merge 10 commits into
Conversation
google-oss-prow
Bot
added
area/ci
aojea
changed the title
|
/ok-to-test |
|
|
||
| // generateGatewayAPIHTTPRoute generates an HTTPRoute for a Workspace using the Gateway API | ||
| func (r *WorkspaceReconciler) generateGatewayAPIHTTPRoute(workspace *kubefloworgv1beta1.Workspace, workspaceKind *kubefloworgv1beta1.WorkspaceKind, service *corev1.Service, imageConfigSpec kubefloworgv1beta1.ImageConfigSpec) *gatewayv1.HTTPRoute { | ||
| namePrefix := generateNamePrefix(workspace.Name, maxVirtualServiceNameLength) |
There was a problem hiding this comment.
rename maxVirtualServiceNameLength
| } | ||
|
|
||
| // generateGatewayAPIHTTPRoute generates an HTTPRoute for a Workspace using the Gateway API | ||
| func (r *WorkspaceReconciler) generateGatewayAPIHTTPRoute(workspace *kubefloworgv1beta1.Workspace, workspaceKind *kubefloworgv1beta1.WorkspaceKind, service *corev1.Service, imageConfigSpec kubefloworgv1beta1.ImageConfigSpec) *gatewayv1.HTTPRoute { |
There was a problem hiding this comment.
how about moving this into helper package, also generateVirtualServiceHTTPRoute under separate files? so that it is easier to test comprehensively and compare the differences.
|
This is great! |
| skipIstioInstall = os.Getenv("ISTIO_INSTALL_SKIP") == "true" | ||
| isIstioAlreadyInstalled = false | ||
|
|
||
| routingProvider = func() string { |
There was a problem hiding this comment.
set skipIstioInstall = True when ROUTING_PROVIDER != "istio"?
And add a check if gateway-api is installed when ROUTING_PROVIDER == 'gateway-api'
aojea
force-pushed
the
gateway_api
branch
4 times, most recently
from
August 5, 2026 14:25
b455568 to
c91b055
Compare
Member
|
@aojea FYI that your commits are showing up as "unverified" 😅 |
Author
sorry , trying to get it in a better shape for reviewing |
Member
|
/retitle feat: implement Gateway API routing provider |
google-oss-prow
Bot
changed the title
aojea
changed the title
aojea
marked this pull request as draft
August 18, 2026 17:34
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
aojea
force-pushed
the
gateway_api
branch
3 times, most recently
from
September 3, 2026 17:44
7c5412b to
2be1536
Compare
Introduce a routing-provider abstraction (none, istio, gateway-api) so workspace routes can be published without Istio. The gateway-api provider generates an HTTPRoute per Workspace, mirroring the VirtualService flow. The istio provider and the legacy --use-istio flag behave exactly as before; new configuration lives in cmd/standalone.go and config/environment_standalone.go to keep the upstream files stable.
…workPolicy Workspace pods serve an unauthenticated notebook, so without a reachability control any pod in the cluster can reach them directly, bypassing the routing layer. In an Istio deployment this is covered by the per-namespace AuthorizationPolicy created by the Kubeflow Profile controller; nothing covered it otherwise. When --workspace-network-policy is set, the controller generates a NetworkPolicy per Workspace allowing ingress only from the configured routing-layer pods. Selecting the pod is enough to deny everything else, so no default-deny policy is needed in the workspace namespace.
A new component that authenticates callers against any OIDC issuer and
authorizes them against Kubernetes RBAC with SubjectAccessReview. It
replaces both things the Kubeflow distribution provided: the identity
header injected by oauth2-proxy, and the per-namespace Istio
AuthorizationPolicy that bound a workspace to its owner.
The service speaks both protocols named by the Gateway API ExternalAuth
filter (GEP-1494): Envoy ext_authz gRPC and forward-auth HTTP. Istio's
AuthorizationPolicy with action CUSTOM can call the same gRPC endpoint,
so one deployment covers both routing providers.
Requests under /workspace/connect/{ns}/{name}/ require get on the target
Workspace; other paths require only a valid identity. Authorized requests
carry the ID token as a bearer token so upstreams can revalidate it, and
identity headers are overwritten so a client-supplied value never survives.
Workspace routes are proxied straight to a notebook pod, so without this filter nothing authenticates or authorizes a request before it reaches the workspace. When --external-auth-backend-name is set, generated HTTPRoutes carry an ExternalAuth filter (GEP-1494) pointing at the authorization service, placed before any URLRewrite so the service sees the path the client requested. An invalid or partial configuration is rejected at startup rather than emitting routes the data plane would refuse.
The header authenticator trusts kubeflow-userid unconditionally: anything that can reach the backend port can assert any identity. When ENABLE_TOKEN_AUTH is set, a request carrying a bearer token is instead authenticated with a TokenReview, so the identity comes from the API server and agrees with what an administrator names in a RoleBinding. Bearer tokens are handled separately from the header authenticator rather than chained through a union: a union treats a rejected token as unauthenticated and falls through, which would let a caller present a garbage token alongside a header they control. TokenReview also validates ServiceAccount tokens, so in-cluster clients gain real authentication even without OIDC on the API server. Off by default because it requires the API server to trust the same OIDC issuer as the gateway.
GetNamespaces returned every namespace the backend ServiceAccount can see, unfiltered, and required cluster-wide list. In the Kubeflow distribution this was masked by the Central Dashboard supplying the namespace picker; standalone users would receive everything or a 403. Evaluate a SubjectAccessReview per namespace (bounded concurrency via errgroup) and return only the namespaces in which the caller can get workspaces.
Components and overlays to deploy the controller, backend and frontend behind a Gateway API Gateway instead of Istio: - gateway-api components route each service through HTTPRoutes and restrict ingress with NetworkPolicies selecting the gateway pods, not the whole namespace - the workspace-network-policy component enables the per-workspace NetworkPolicy in the controller - the ext-authz component and the gateway-api-ext-authz overlay wire the controller to the external authorization service The istio overlays are unchanged and remain the default.
Lets the build use a newer toolchain than the base image when go.mod requires one.
…the Tilt workflow Adds a ROUTING_PROVIDER=gateway-api mode to the Tiltfile, deploying the Gateway API CRDs, a Gateway backed by cloud-provider-kind, Dex as a local OIDC issuer, and the workspaces-authz service, so the full standalone authentication path can be exercised locally instead of the devAuth localStorage shim.
The e2e suite honours ROUTING_PROVIDER (defaulting to istio) and skips the Istio install for other providers. With ENABLE_EXT_AUTHZ=true it asserts that workspace routes reject unauthenticated requests, so a regression that drops the ExternalAuth filter fails the suite.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This commit introduces Gateway API (v1) support as an alternative to Istio for routing traffic to workspaces.
Key changes:
This allows Kubeflow users to adopt any Gateway API-compliant ingress controller (like Envoy Gateway) instead of being strictly coupled to Istio.
Assisted by AI