*: use a Go binary for image mirroring - #257
*: use a Go binary for image mirroring#257Steve Kuznetsov (stevekuznetsov) wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces a new Go-based imagemirror tool (with subcommands to mirror images from a registry or from an OCI layout tar) and updates pipeline step resolution/schema to optionally use a native imagemirror binary instead of the existing on-demand shell script approach.
Changes:
- Added a new
tools/imagemirrorGo module implementingfrom-registryandfrom-oci-layoutmirroring flows (including ACR token exchange and optional Key Vault pull-secret auth). - Extended
pipelines/types.ImageMirrorStepand the pipeline schema to supportuseNativeMirror, and added resolution logic to emit a directimagemirror ...command. - Updated tests/fixtures and workspace configuration (
go.work, sums) to include the new tool and validate the resolved native commands.
Reviewed changes
Copilot reviewed 13 out of 14 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/imagemirror/go.mod | New module definition and direct dependencies for the imagemirror tool. |
| tools/imagemirror/go.sum | New module dependency checksums. |
| tools/imagemirror/command.go | Defines the root cobra command and wires subcommands. |
| tools/imagemirror/fromregistry.go | Implements mirroring from a source registry (supports anonymous, docker config, or KV pull-secret auth). |
| tools/imagemirror/fromocilayout.go | Implements mirroring from a local OCI-layout tar using metadata-provided tags. |
| tools/imagemirror/acr.go | Implements ARM→ACR token exchange with retry and minimal JWT expiry parsing. |
| pipelines/types/pipeline.schema.v1.json | Adds useNativeMirror to the ImageMirror step schema. |
| pipelines/types/imagemirror.go | Adds UseNativeMirror flag and resolves native vs script-based ShellStep commands. |
| pipelines/types/imagemirror_test.go | Extends test cases to cover native mirroring resolution. |
| pipelines/types/testdata/zz_fixture_*.yaml | New golden fixtures for native mirroring command resolution. |
| go.work | Adds ./tools/imagemirror to the workspace. |
| go.work.sum | Updates workspace sums for new dependencies. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
4f9cc7a to
7f1061e
Compare
| case "oci-layout": | ||
| variables = []Variable{ | ||
| namedVariable("TARGET_ACR", input.TargetACR), | ||
| namedVariable("REPOSITORY", input.Repository), | ||
| namedVariable("IMAGE_TAR", input.ImageTarFileName), | ||
| namedVariable("IMAGE_METADATA", input.ImageMetadataFileName), | ||
| } | ||
| parts := []string{ | ||
| opts.ImageMirrorBinary, "from-oci-layout", | ||
| "--target-acr", "${TARGET_ACR}", | ||
| "--acr-suffix", opts.ACRSuffix, | ||
| "--repository", "${REPOSITORY}", | ||
| "--image-tar", "${IMAGE_TAR}", | ||
| "--image-metadata", "${IMAGE_METADATA}", | ||
| "--cloud", opts.Cloud, | ||
| } | ||
| command = strings.Join(parts, " ") |
| vaultURI := fmt.Sprintf("https://%s.vault.azure.net", vaultName) | ||
| client, err := azsecrets.NewClient(vaultURI, cred, &azsecrets.ClientOptions{ | ||
| ClientOptions: azcore.ClientOptions{Cloud: cloudConfig}, | ||
| }) |
| var config dockerConfig | ||
| if err := json.Unmarshal(configData, &config); err != nil { | ||
| return fmt.Errorf("failed to unmarshal Docker auth config: %w", err) | ||
| } | ||
|
|
||
| for registry, entry := range config.Auths { | ||
| decoded, err := base64.StdEncoding.DecodeString(entry.Auth) | ||
| if err != nil { | ||
| return fmt.Errorf("failed to decode auth for registry %s: %w", registry, err) | ||
| } | ||
|
|
||
| parts := strings.SplitN(string(decoded), ":", 2) | ||
| if len(parts) != 2 { | ||
| return fmt.Errorf("invalid auth format for registry %s", registry) | ||
| } | ||
|
|
||
| cred := auth.Credential{ | ||
| Username: parts[0], | ||
| Password: parts[1], | ||
| } | ||
| if err := store.Put(ctx, registry, cred); err != nil { | ||
| return fmt.Errorf("failed to store credential for registry %s: %w", registry, err) | ||
| } | ||
| } |
7f1061e to
7b2d60f
Compare
Signed-off-by: Steve Kuznetsov <stekuznetsov@microsoft.com>
7b2d60f to
9587d97
Compare
| switch input.CopyFrom { | ||
| case "oci-layout": | ||
| variables = []Variable{ | ||
| namedVariable("TARGET_ACR", input.TargetACR), | ||
| namedVariable("REPOSITORY", input.Repository), | ||
| namedVariable("IMAGE_TAR", input.ImageTarFileName), | ||
| namedVariable("IMAGE_METADATA", input.ImageMetadataFileName), | ||
| } | ||
| parts := []string{ | ||
| opts.ImageMirrorBinary, "from-oci-layout", | ||
| "--target-acr", "${TARGET_ACR}", | ||
| "--acr-suffix", opts.ACRSuffix, | ||
| "--repository", "${REPOSITORY}", | ||
| "--image-tar", "${IMAGE_TAR}", | ||
| "--image-metadata", "${IMAGE_METADATA}", | ||
| "--cloud", opts.Cloud, | ||
| } | ||
| command = strings.Join(parts, " ") | ||
| default: |
| // fetchPullSecretFromKeyVault fetches a pull secret from Azure Key Vault and base64-decodes it. | ||
| func fetchPullSecretFromKeyVault(ctx context.Context, cred azcore.TokenCredential, cloudConfig cloud.Configuration, vaultName, secretName string) ([]byte, error) { | ||
| vaultURI := fmt.Sprintf("https://%s.vault.azure.net", vaultName) | ||
| client, err := azsecrets.NewClient(vaultURI, cred, &azsecrets.ClientOptions{ | ||
| ClientOptions: azcore.ClientOptions{Cloud: cloudConfig}, | ||
| }) |
|
PR needs rebase. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
Steve Kuznetsov (@stevekuznetsov) #321 adds Different consumers — yours copies in-process via ORAS so it never writes an The duplicated part is just the token exchange. Happy to go either way:
Any preference? Also happy to help get this rebased if it's just been waiting. |
|
Wenqi He (@weherdh) you can land yours first - I am not sure if I have time yet to finish pushing this, and can follow up |
No description provided.