feat: dynamic knative service orchestration for base model registration - #529
feat: dynamic knative service orchestration for base model registration#529yshmael wants to merge 5 commits into
Conversation
… routing with /predict path
| api_instance = client.CustomObjectsApi() | ||
|
|
||
| # Knative API Group definitions | ||
| group = "serving.knative.dev" |
There was a problem hiding this comment.
Kindly read this from env variable with default value !
|
|
||
| # 3. Load and populate the YAML template | ||
| # Adjust the path based on where this script runs relative to project root | ||
| template_path = os.path.join( |
There was a problem hiding this comment.
use python relative import !
| manifest_str = file.read() | ||
|
|
||
| # Determine dynamic image - assuming a base model server image for now | ||
| image_url = os.getenv("MODEL_SERVER_IMAGE", "ghcr.io/hotosm/fair-model-server:latest") |
There was a problem hiding this comment.
I am not sure this thing exists yet , and why this is needed ? if needed the env variable needs to be defined as clear self explanatory name so it won't create confusion , kindly read it from settings.variable andread it from env variable as well if the use of this is justified !
| ) | ||
| except ApiException as e: | ||
| if e.status == 409: # Conflict: Service already exists, patch it instead | ||
| api_instance.patch_namespaced_custom_object( |
There was a problem hiding this comment.
patch doesn't create if the resources doesn't exist ?
| time.sleep(1) | ||
|
|
||
| # Fallback to predictable internal cluster URL if polling times out | ||
| return f"http://{model_name}.{namespace}.svc.cluster.local" |
There was a problem hiding this comment.
we can't have this fallback it should strictly be passing
| # 5. Fetch the resulting Knative URL | ||
| # Knative takes a moment to assign the URL, we implement a brief polling loop | ||
| for _ in range(10): | ||
| try: |
There was a problem hiding this comment.
this try except block should not be here and also the for loop for 10 seconds , there should be some function in upstream we can use ?
|
|
||
| try: | ||
| # 3. Dynamic namespace detection based on Django environment | ||
| namespace = "fair-prod" if not settings.DEBUG else "fair-staging" |
There was a problem hiding this comment.
env variable from settings ,
|
|
||
| # 5. Construct the exact inference URL for the deployed model | ||
| # Using the sanitized model_name to ensure valid DNS formatting | ||
| expected_predict_url = f"https://{model_name}.predict.ai.hotosm.org/predict" |
There was a problem hiding this comment.
also the env variable for the prediction url
| expected_predict_url = f"https://{model_name}.predict.ai.hotosm.org/predict" | ||
|
|
||
| # 6. Save the correctly formatted endpoint into the STAC properties | ||
| stac_item.setdefault("properties", {})["knative_endpoint"] = expected_predict_url |
There was a problem hiding this comment.
knative_endpoint doesn't exist in stac it would be mlm:inference-endpoint
| stac_item.setdefault("properties", {})["knative_endpoint"] = expected_predict_url | ||
|
|
||
| # 7. Register the Knative URL as the STAC mlm:inference-endpoint asset | ||
| if not data.get("inference_endpoint"): |
There was a problem hiding this comment.
it shoudl always override ! so this is not needed
|
|
||
| # ---> KNATIVE DEPLOYMENT BLOCK START <--- | ||
| # 1. Fetch the STAC name (fallback to fair-base-model if missing) | ||
| stac_name = stac_item.get("properties", {}).get("mlm:name", "fair-base-model") |
There was a problem hiding this comment.
no , not needed , regiseration is always for basemodel , local models are not registered !
3 participants
Description
Addresses the workflow requested by @Kshitij to fully automate Knative service deployment dynamically whenever a new base model is registered via the
/api/v1/base-models/endpoint.This relies on the
fair-model-deployerRBAC permissions recently merged intok8s-infra. The Django API now safely acts as an in-cluster operator using the Kubernetes Python SDK.Technical Changes
kubernetesto backend dependencies viauv.infra/knative-model-template.yamlas the foundational Knative configuration.knative_deployer.py): Added a utility to securely hook into EKS viaload_incluster_config()and dynamically create/patch Knative CRDs viaCustomObjectsApi.POST /base-models/endpoint inviews.pyto trigger the deployment. Added a "smart fallback" logic to automatically construct and inject themlm:inference-endpointSTAC URL ashttps://<model_name>.predict.ai.hotosm.org/predict.deploy_model_to_knativefunction intest_base_model_endpoints.pyto ensure the automated GitHub Actions CI pipeline passes without requiring a live cluster connection. Additionally, updated the test suite to expect the new auto-generated Knative URL instead of a blank field when no endpoint is explicitly provided.