Repository navigation
Modernize code samples and add deployment E2E CI #6748
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
86d4d13
cb40edb
18d3e24
08b49aa
08ee56c
60cce3b
84e5674
9eda5b5
fcbc7b2
4dac3db
69278d6
fb30c69
74dc042
ef15d37
24c54ca
cbc1ac5
6586c4c
128b510
2de8a2b
56ec0bf
54cccfb
fd24296
7ee539f
7bfc8aa
9de94b1
17037fa
adf0d1e
ec9c81e
4b6148d
9c153e0
c852e25
776e951
3f8f33c
7367e15
c3d8633
c79ebc3
d5be777
6097f21
4e2a68f
2e13c21
fee5f0c
4d924c3
35790a4
20abec2
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| name: samples-test | ||
| on: | ||
| pull_request: | ||
| paths: ['code-samples/**', '.github/workflows/samples-test.yaml'] | ||
| permissions: | ||
| contents: read | ||
| jobs: | ||
| discover: | ||
| runs-on: ubuntu-24.04 | ||
| timeout-minutes: 10 | ||
| outputs: | ||
| directories: ${{ steps.samples.outputs.directories }} | ||
| steps: | ||
| - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 | ||
| with: | ||
| ref: ${{ github.event.pull_request.head.sha }} | ||
| fetch-depth: 0 | ||
| persist-credentials: false | ||
| - id: samples | ||
| name: Find changed sample directories | ||
| run: | | ||
| diff_range="${{ github.event.pull_request.base.sha }}...HEAD" | ||
| directories=() | ||
| while IFS= read -r -d '' script; do | ||
| directory=${script%/ci.sh} | ||
| changed=$(git diff --name-only --no-renames "$diff_range" -- "$directory/") | ||
| if [[ -n "$changed" ]]; then | ||
| directories+=("$directory") | ||
| fi | ||
| done < <(git ls-files -z -- 'code-samples/**/ci.sh') | ||
| echo 'Directories without ci.sh are NOT TESTED.' >> "$GITHUB_STEP_SUMMARY" | ||
| # Use consistent section/sample names without collision detection. | ||
| matrix=$( | ||
| for directory in "${directories[@]}"; do | ||
| relative=${directory#code-samples/} | ||
| section=${relative%%/*} | ||
| name="$section/${directory##*/}" | ||
|
|
||
| jq -cn \ | ||
| --arg directory "$directory" \ | ||
| --arg name "$name" \ | ||
| '{directory: $directory, name: $name}' | ||
| done | jq -sc 'unique_by(.directory)' | ||
| ) | ||
| echo "directories=$matrix" >> "$GITHUB_OUTPUT" | ||
|
|
||
| test: | ||
| name: ${{ matrix.name }} | ||
| needs: discover | ||
| if: needs.discover.outputs.directories != '[]' | ||
| runs-on: ubuntu-24.04 | ||
| timeout-minutes: 30 | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| include: ${{ fromJSON(needs.discover.outputs.directories) }} | ||
| steps: | ||
| - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 | ||
| with: | ||
| ref: ${{ github.event.pull_request.head.sha }} | ||
| persist-credentials: false | ||
| - uses: chainguard-dev/actions/setup-kind@8f31e9482b447ff2c69eca944a09b4b31a4a85d0 | ||
| with: | ||
| k8s-version: 1.36.x | ||
|
Comment on lines
+55
to
+64
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We're setting up a
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oddly i agree but it seems like github is handling well - eg. see checks in this PR it is running all of the code-sample e2e tests. Those are really just a light smoke test that the docker file builds and it can deploy. I was expecting dependabot updates to really trigger one or two code-samples at most. |
||
| - name: Install latest Knative Serving YAMLs | ||
| run: | | ||
| kubectl apply -f https://github.com/knative/serving/releases/latest/download/serving-crds.yaml | ||
| kubectl apply -f https://github.com/knative/serving/releases/latest/download/serving-core.yaml | ||
| kubectl apply -f https://github.com/knative-extensions/net-kourier/releases/latest/download/kourier.yaml | ||
| kubectl patch cm config-network -n knative-serving --type merge -p '{"data":{"ingress-class":"kourier.ingress.networking.knative.dev"}}' | ||
| kubectl patch cm config-domain -n knative-serving --type merge -p '{"data":{"example.test":""}}' | ||
| kubectl wait deployment --all -n knative-serving --for=condition=Available --timeout=300s | ||
| kubectl wait deployment --all -n kourier-system --for=condition=Available --timeout=300s | ||
| - name: Test | ||
| working-directory: ${{ matrix.directory }} | ||
| run: | | ||
| kubectl -n kourier-system port-forward service/kourier 18080:80 & | ||
| forward=$! | ||
| trap 'kill "$forward" 2>/dev/null || true' EXIT | ||
| for _attempt in {1..30}; do | ||
| (true > /dev/tcp/127.0.0.1/18080) 2>/dev/null && break | ||
| sleep 1 | ||
| done | ||
| (true > /dev/tcp/127.0.0.1/18080) 2>/dev/null | ||
| source ./ci.sh | ||
| main | ||
| - name: Failure diagnostics | ||
| if: failure() | ||
| run: | | ||
| kubectl get ksvc,revisions,pods -A -o wide || true | ||
| kubectl get events -A --sort-by=.metadata.creationTimestamp || true | ||
| kubectl -n default logs --all-containers -l serving.knative.dev/service --tail=100 || true | ||
| - name: Delete disposable cluster | ||
| if: always() | ||
| run: kind delete cluster | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,17 +1,11 @@ | ||
| # Use the official Clojure image. | ||
| # https://hub.docker.com/_/clojure | ||
| FROM clojure | ||
|
|
||
| # Create the project and download dependencies. | ||
| WORKDIR /usr/src/app | ||
| FROM clojure:temurin-25-lein-2.13.0-noble AS builder | ||
| WORKDIR /app | ||
| COPY project.clj . | ||
| RUN lein deps | ||
| COPY src ./src | ||
| RUN lein uberjar | ||
|
|
||
| # Copy local code to the container image. | ||
| COPY . . | ||
|
|
||
| # Build an uberjar release artifact. | ||
| RUN mv "$(lein uberjar | sed -n 's/^Created \(.*standalone\.jar\)/\1/p')" app-standalone.jar | ||
|
|
||
| # Run the web service on container startup. | ||
| CMD ["java", "-jar", "app-standalone.jar"] | ||
| FROM eclipse-temurin:25-jre-noble | ||
| WORKDIR /app | ||
| COPY --from=builder /app/target/helloworld.jar . | ||
| CMD ["java", "-jar", "helloworld.jar"] |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,154 +1,44 @@ | ||
| # Hello World - Clojure | ||
|
|
||
| A simple web app written in Clojure that you can use for testing. It reads in an | ||
| env variable `TARGET` and prints "Hello \${TARGET}!". If TARGET is not | ||
| specified, it will use "World" as the TARGET. | ||
| A Clojure HTTP service using Ring and Jetty on Java 25. It returns | ||
| `Hello ${TARGET}!` followed by a newline. `TARGET` defaults to `World`, and | ||
| `PORT` defaults to `8080`. | ||
|
|
||
| ## Prerequisites | ||
| The [source](src/helloworld/core.clj) and [dependencies](project.clj) are built | ||
| into an executable jar by the [Dockerfile](Dockerfile); the runtime image | ||
| contains only a Java 25 JRE and the application jar. | ||
|
|
||
| - A Kubernetes cluster with Knative installed and DNS configured. Follow the | ||
| [Knative installation instructions](https://knative.dev/docs/install/) if you need to create | ||
| one. | ||
| - [Docker](https://www.docker.com) installed and running on your local machine, | ||
| and a Docker Hub account configured (we'll use it for a container registry). | ||
| ## Build and run locally | ||
|
|
||
| ## Recreating the sample code | ||
| From this directory, with Docker running: | ||
|
|
||
| While you can clone all of the code from this directory, hello world apps are | ||
| generally more useful if you build them step-by-step. The following instructions | ||
| recreate the source files from this folder. | ||
|
|
||
| 1. Create a new file named `src/helloworld/core.clj` and paste the following | ||
| code. This code creates a basic web server which listens on port 8080: | ||
|
|
||
| ```clojure | ||
| (ns helloworld.core | ||
| (:use ring.adapter.jetty) | ||
| (:gen-class)) | ||
|
|
||
| (defn handler [request] | ||
| {:status 200 | ||
| :headers {"Content-Type" "text/html"} | ||
| :body (str "Hello " | ||
| (if-let [target (System/getenv "TARGET")] | ||
| target | ||
| "World") | ||
| "!\n")}) | ||
|
|
||
| (defn -main [& args] | ||
| (run-jetty handler {:port (if-let [port (System/getenv "PORT")] | ||
| (Integer/parseInt port) | ||
| 8080)})) | ||
| ``` | ||
|
|
||
| 1. In your project directory, create a file named `project.clj` and copy the | ||
| code below into it. This code defines the project's dependencies and | ||
| entrypoint. | ||
|
|
||
| ```clojure | ||
| (defproject helloworld "1.0.0-SNAPSHOT" | ||
| :description "Hello World - Clojure sample" | ||
| :dependencies [[org.clojure/clojure "1.9.0"] | ||
| [ring/ring-core "1.6.3"] | ||
| [ring/ring-jetty-adapter "1.6.3"]] | ||
| :main helloworld.core) | ||
| ``` | ||
|
|
||
| 1. In your project directory, create a file named `Dockerfile` and copy the code | ||
| block below into it. For detailed instructions on dockerizing a Clojure app, | ||
| see | ||
| [the clojure image documentation](https://github.com/docker-library/docs/tree/master/clojure). | ||
|
|
||
| ```docker | ||
| # Use the official Clojure image. | ||
| # https://hub.docker.com/_/clojure | ||
| FROM clojure | ||
|
|
||
| # Create the project and download dependencies. | ||
| WORKDIR /usr/src/app | ||
| COPY project.clj . | ||
| RUN lein deps | ||
|
|
||
| # Copy local code to the container image. | ||
| COPY . . | ||
|
|
||
| # Build an uberjar release artifact. | ||
| RUN mv "$(lein uberjar | sed -n 's/^Created \(.*standalone\.jar\)/\1/p')" app-standalone.jar | ||
|
|
||
| # Run the web service on container startup. | ||
| CMD ["java", "-jar", "app-standalone.jar"] | ||
| ``` | ||
|
|
||
| 1. Create a new file, `service.yaml` and copy the following service definition | ||
| into the file. Make sure to replace `{username}` with your Docker Hub | ||
| username. | ||
|
|
||
| ```yaml | ||
| apiVersion: serving.knative.dev/v1 | ||
| kind: Service | ||
| metadata: | ||
| name: helloworld-clojure | ||
| namespace: default | ||
| spec: | ||
| template: | ||
| spec: | ||
| containers: | ||
| - image: docker.io/{username}/helloworld-clojure | ||
| env: | ||
| - name: TARGET | ||
| value: "Clojure Sample v1" | ||
| ``` | ||
|
|
||
| ## Building and deploying the sample | ||
|
|
||
| Once you have recreated the sample code files (or used the files in the sample | ||
| folder) you're ready to build and deploy the sample app. | ||
|
|
||
| 1. Use Docker to build the sample code into a container. To build and push with | ||
| Docker Hub, run these commands replacing `{username}` with your Docker Hub | ||
| username: | ||
|
|
||
| ```bash | ||
| # Build and push the container on your local machine. | ||
| docker buildx build --platform linux/arm64,linux/amd64 -t "{username}/helloworld-clojure" --push . | ||
| ``` | ||
|
|
||
| 1. After the build has completed and the container is pushed to docker hub, you | ||
| can deploy the app into your cluster. Ensure that the container image value | ||
| in `service.yaml` matches the container you built in the previous step. Apply | ||
| the configuration using `kubectl`: | ||
|
|
||
| ```bash | ||
| kubectl apply --filename service.yaml | ||
| ``` | ||
|
|
||
| 1. Now that your service is created, Knative will perform the following steps: | ||
|
|
||
| - Create a new immutable revision for this version of the app. | ||
| - Network programming to create a route, ingress, service, and load balance | ||
| for your app. | ||
| - Automatically scale your pods up and down (including to zero active pods). | ||
|
|
||
| 1. To find the URL for your service, use | ||
| ```bash | ||
| docker build -t helloworld-clojure . | ||
| docker run --rm -p 8080:8080 helloworld-clojure | ||
| # In another terminal: | ||
| curl http://localhost:8080 | ||
| # Hello World! | ||
| ``` | ||
|
|
||
| ``` | ||
| kubectl get ksvc helloworld-clojure --output=custom-columns=NAME:.metadata.name,URL:.status.url | ||
| NAME URL | ||
| helloworld-clojure http://helloworld-clojure.default.1.2.3.4.sslip.io | ||
| ``` | ||
| Set `TARGET` or `PORT` with Docker's `-e` option. If you change `PORT`, also | ||
| update the container port in your `-p` mapping. | ||
|
|
||
| 1. Now you can make a request to your app and see the result. Replace | ||
| the URL below with the URL returned in the previous command. | ||
| ## Deploy to Knative | ||
|
|
||
| ```bash | ||
| curl http://helloworld-clojure.default.1.2.3.4.sslip.io | ||
| Hello World! | ||
| ``` | ||
| Use a cluster with [Knative Serving](https://knative.dev/docs/install/) and | ||
| networking configured, and a Docker Hub account. Replace `{username}` below | ||
| and in [service.yaml](service.yaml) with your Docker Hub username. | ||
|
|
||
| ## Removing the sample app deployment | ||
| ```bash | ||
| docker buildx build --platform linux/arm64,linux/amd64 -t "{username}/helloworld-clojure" --push . | ||
| kubectl apply -f service.yaml | ||
| kubectl wait --for=condition=Ready service.serving.knative.dev/helloworld-clojure --timeout=300s | ||
| curl "$(kubectl get ksvc helloworld-clojure -o jsonpath='{.status.url}')" | ||
| # Hello Clojure Sample v1! | ||
| ``` | ||
|
|
||
| To remove the sample app from your cluster, delete the service record: | ||
| The manifest sets `TARGET` to `Clojure Sample v1`. Remove the sample with: | ||
|
|
||
| ```bash | ||
| kubectl delete --filename service.yaml | ||
| kubectl delete -f service.yaml | ||
| ``` |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| #!/usr/bin/env bash | ||
| # Source from this sample directory; the caller owns cluster setup and ingress forwarding. | ||
| IMAGE=dev.local/helloworld-clojure:ci | ||
| SAMPLE_URL=http://127.0.0.1:18080 | ||
|
|
||
| main() ( | ||
| set -euo pipefail | ||
| build | ||
| deploy | ||
| test | ||
| ) | ||
|
|
||
| build() ( | ||
| set -euo pipefail | ||
| docker build -t "$IMAGE" . | ||
| ) | ||
|
|
||
| deploy() ( | ||
| set -euo pipefail | ||
| kind load docker-image "$IMAGE" | ||
| kubectl patch --local -f service.yaml --type=json -p "$( | ||
| jq -cn --arg image "$IMAGE" '[ | ||
| {op: "replace", path: "/spec/template/spec/containers/0/image", value: $image}, | ||
| {op: "add", path: "/spec/template/spec/containers/0/imagePullPolicy", value: "IfNotPresent"} | ||
| ]' | ||
| )" -o yaml | kubectl --namespace default apply -f - | ||
| kubectl --namespace default wait -f service.yaml --for=condition=Ready --timeout=300s | ||
| ) | ||
|
|
||
| test() ( | ||
| set -euo pipefail | ||
| local service_url service_host | ||
| service_url=$(kubectl --namespace default get -f service.yaml -o jsonpath='{.status.url}') | ||
| service_host=${service_url#*://} | ||
| [[ "$service_host" =~ ^[a-zA-Z0-9.-]+$ ]] | ||
| local body | ||
| body=$(curl -fsS --max-time 30 \ | ||
| -H "Host: $service_host" "$SAMPLE_URL/") | ||
| [[ $body == 'Hello Clojure Sample v1!' ]] | ||
| ) | ||
|
dprotaso marked this conversation as resolved.
Resolved
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It feels like we need some meta-documentation (CONTRIBUTING.md or docs/contributing) about how this CI works. If I'm reading it correctly, it spins up a Kind cluster for each sample, loads serving (but not eventing) into the cluster, and then runs a CI script with a port-forwarder to see if whatever is loaded into the cluster is responding / working.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It should also install eventing for the samples that need it (eg. eventing - python example)
Let me revisit that
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
eventing-python is actually the only leftover sample that needs Eventing deployed?