Skip to content
This repository was archived by the owner on Jul 18, 2025. It is now read-only.

Commit 60d6d5a

Browse files
author
Ian Campbell
committed
ci: run e2e tests via gotestsum and record junit result
As well as gotestsum we now also need to build `test2json` because test binaries build with `go test -c` do not support the `-test` option. That option just wraps the test run in the `test2json` code which translates from the standard output syntax to json. (See golang/go#31051 for more info). This uses `cmd/test2json` the source of which is part of the go distribution and is therefore available in the base images. Once we have them then we "simply" chain them together, `gotestsum` takes a `--raw-command` for which we pass `test2json`. In turn `test2json` takes a command which it will wrap and we pass the actual e2e test binary. Running `test2json` manually gives us the opportunity to set the package name (with `-p`) we give it a distinct name rather than munging the output after the fact like we do with the unit tests. Since the only references to these tools are in `Jenkinsfile.*` we don't need to worry about making them available for non-containerised runs. `test2json` requires verbose output, so add `-test.v` in one case. Signed-off-by: Ian Campbell <ijc@docker.com>
1 parent 4cb441b commit 60d6d5a

4 files changed

Lines changed: 27 additions & 5 deletions

File tree

‎Dockerfile‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,11 @@ RUN mkdir $GOPATH/src/gotest.tools && \
3030
GO111MODULE=on GOOS=darwin go build -o /usr/local/bin/gotestsum-darwin gotest.tools/gotestsum && \
3131
GO111MODULE=on GOOS=windows go build -o /usr/local/bin/gotestsum-windows.exe gotest.tools/gotestsum && \
3232
ln -s gotestsum-linux /usr/local/bin/gotestsum
33+
# Source for cmd/test2json is part of the Go distribution and is
34+
# therefore available in the base image.
35+
RUN GOOS=linux go build -o /usr/local/bin/test2json-linux cmd/test2json && \
36+
GOOS=darwin go build -o /usr/local/bin/test2json-darwin cmd/test2json && \
37+
GOOS=windows go build -o /usr/local/bin/test2json-windows.exe cmd/test2json
3338
RUN go get -d gopkg.in/mjibson/esc.v0 && \
3439
cd /go/src/github.com/mjibson/esc && \
3540
go build -v -o /usr/bin/esc . && \

‎Jenkinsfile‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -185,11 +185,13 @@ pipeline {
185185
dir('e2e'){
186186
unstash "e2e"
187187
}
188-
sh './docker-app-e2e-linux --e2e-path=e2e'
188+
sh './gotestsum-linux --format short-verbose --junitfile e2e-linux.xml --raw-command -- ./test2json-linux -t -p docker-app/e2e-linux ./docker-app-e2e-linux -test.v --e2e-path=e2e'
189189
}
190190
}
191191
post {
192192
always {
193+
archiveArtifacts 'src/github.com/docker/app/e2e-linux.xml'
194+
junit 'src/github.com/docker/app/e2e-linux.xml'
193195
sh 'docker rmi docker/cnab-app-base:$BUILD_TAG'
194196
deleteDir()
195197
}

‎Jenkinsfile.baguette‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -151,11 +151,13 @@ pipeline {
151151
unstash "e2e"
152152
}
153153
sh './docker-linux version'
154-
sh './docker-app-e2e-linux -test.v --e2e-path=e2e'
154+
sh './gotestsum-linux --format short-verbose --junitfile e2e-linux.xml --raw-command -- ./test2json-linux -t -p docker-app/e2e-linux ./docker-app-e2e-linux -test.v --e2e-path=e2e'
155155
}
156156
}
157157
post {
158158
always {
159+
archiveArtifacts 'src/github.com/docker/app/e2e-linux.xml'
160+
junit 'src/github.com/docker/app/e2e-linux.xml'
159161
sh 'docker rmi docker/cnab-app-base:$BUILD_TAG'
160162
deleteDir()
161163
}
@@ -184,11 +186,13 @@ pipeline {
184186
unstash "e2e"
185187
}
186188
sh './docker-darwin version'
187-
sh './docker-app-e2e-darwin -test.v --e2e-path=e2e'
189+
sh './gotestsum-darwin --format short-verbose --junitfile e2e-darwin.xml --raw-command -- ./test2json-darwin -t -p docker-app/e2e-darwin ./docker-app-e2e-darwin -test.v --e2e-path=e2e'
188190
}
189191
}
190192
post {
191193
always {
194+
archiveArtifacts 'src/github.com/docker/app/e2e-darwin.xml'
195+
junit 'src/github.com/docker/app/e2e-darwin.xml'
192196
sh 'docker rmi docker/cnab-app-base:$BUILD_TAG'
193197
deleteDir()
194198
}
@@ -217,11 +221,13 @@ pipeline {
217221
dir('e2e'){
218222
unstash "e2e"
219223
}
220-
bat 'docker-app-e2e-windows.exe -test.v --e2e-path=e2e'
224+
bat 'gotestsum-windows.exe --format short-verbose --junitfile e2e-windows.xml --raw-command -- test2json-windows.exe -t -p docker-app/e2e-windows docker-app-e2e-windows.exe -test.v --e2e-path=e2e'
221225
}
222226
}
223227
post {
224228
always {
229+
archiveArtifacts 'src/github.com/docker/app/e2e-windows.xml'
230+
junit 'src/github.com/docker/app/e2e-windows.xml'
225231
bat 'docker rmi docker/cnab-app-base:%BUILD_TAG%'
226232
deleteDir()
227233
}

‎docker.Makefile‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,10 +71,19 @@ e2e-cross: create_bin
7171
docker cp $(E2E_CROSS_CTNR_NAME):$(PKG_PATH)/bin/$(BIN_NAME)-e2e-linux bin/$(BIN_NAME)-e2e-linux
7272
docker cp $(E2E_CROSS_CTNR_NAME):$(PKG_PATH)/bin/$(BIN_NAME)-e2e-darwin bin/$(BIN_NAME)-e2e-darwin
7373
docker cp $(E2E_CROSS_CTNR_NAME):$(PKG_PATH)/bin/$(BIN_NAME)-e2e-windows.exe bin/$(BIN_NAME)-e2e-windows.exe
74+
docker cp $(E2E_CROSS_CTNR_NAME):/usr/local/bin/gotestsum-linux bin/gotestsum-linux
75+
docker cp $(E2E_CROSS_CTNR_NAME):/usr/local/bin/gotestsum-darwin bin/gotestsum-darwin
76+
docker cp $(E2E_CROSS_CTNR_NAME):/usr/local/bin/gotestsum-windows.exe bin/gotestsum-windows.exe
77+
docker cp $(E2E_CROSS_CTNR_NAME):/usr/local/bin/test2json-linux bin/test2json-linux
78+
docker cp $(E2E_CROSS_CTNR_NAME):/usr/local/bin/test2json-darwin bin/test2json-darwin
79+
docker cp $(E2E_CROSS_CTNR_NAME):/usr/local/bin/test2json-windows.exe bin/test2json-windows.exe
7480
docker rm $(E2E_CROSS_CTNR_NAME)
7581
@$(call chmod,+x,bin/$(BIN_NAME)-e2e-linux)
7682
@$(call chmod,+x,bin/$(BIN_NAME)-e2e-darwin)
77-
@$(call chmod,+x,bin/$(BIN_NAME)-e2e-windows.exe)
83+
@$(call chmod,+x,bin/gotestsum-linux)
84+
@$(call chmod,+x,bin/gotestsum-darwin)
85+
@$(call chmod,+x,bin/test2json-linux)
86+
@$(call chmod,+x,bin/test2json-darwin)
7887

7988
tars:
8089
tar --transform='flags=r;s|$(BIN_NAME)-linux|$(BIN_NAME)-plugin-linux|' -czf bin/$(BIN_NAME)-linux.tar.gz -C bin $(BIN_NAME)-linux ${BIN_STANDALONE_NAME}-linux

0 commit comments

Comments
 (0)