jobs: add support for one-shot and scheduled jobs

[glours]

What I did
Added support for the jobs: top-level element from the Compose Specification: a job is a one-shot unit of work, distinct from a long-running servic, that fires either manually or on a schedule.

• docker compose up registers a project's scheduled jobs with the
  engine, so they fire on the daemon's own clock from then on,
  independently of the Compose CLI staying attached. Manual jobs are
  registered but never fired by up.
• docker compose run <job> (the same command already used for
  services) triggers a manual job: it starts the job's declared
  dependencies first, streams its output, and exits with the job's
  own exit code, exactly like a one-off service run.
• docker compose down deregisters the project's jobs from the
  engine.
• Re-running up on an unchanged project is a safe no-op. If a job's
  definition changed since it was registered, up (or run) reports
  a clear error asking the user to down the project first rather
  than silently applying the change.

This is an early, working implementation: it targets an experimental engine capability (gated behind a feature flag) that is not yet part of a released Docker Engine, so jobs are not usable against a standard engine today.

Related issue

Based on #14093

(not mandatory) A picture of a cute animal, if possible in relation to what you did

[docker-agent]

🟢 No issues found — LGTM! View logs.

[docker-agent]

Assessment: 🟡 NEEDS ATTENTION

Four medium-severity findings in the new jobs orchestration code: two nil pointer dereference risks (unguarded proto pointer fields after Wait and in createJobRun return paths), one log-streaming race where the container may not yet exist when the goroutine starts, and one typed-error loss on the RunJob CLI exit path.

[docker-agent]

[medium] Log-streaming goroutine starts immediately with a potentially empty ContainerID

The goroutine at line 440 calls s.streamJobLogs(ctx, created.ContainerID, ...) before the container is guaranteed to exist. The RunReply and RunRequest docs state:

ContainerID is set once the container is created.

This means the engine may return a Run record (via CreateAndRun or Run) before the container is created, leaving ContainerID as an empty string at the time streamJobLogs is called. ContainerLogs("") will immediately fail with a "no such container" error. The error is swallowed at logrus.Debugf level and logsDone is closed — the user sees no output from the job even though it runs to completion.

To fix, either:

1. Poll InspectRun until ContainerID is non-empty before starting the goroutine, or
2. Issue a Wait call with WaitConditionRunning first so the container is guaranteed to exist before streaming.

Confidence	Score
🟡 moderate	57/100

[docker-agent]

[medium] Nil pointer dereference: waited.Run is not guarded after jc.Wait() succeeds

After jc.Wait() returns with a nil error (post MapError), run := waited.Run is assigned and immediately dereferenced in switch run.State. The proto-generated WaitReply.Run is a *Run pointer with omitempty — if the engine returns a successful reply with no run field set (e.g., a partial or malformed response), waited.Run is nil, and run.State panics.

MapError only maps gRPC status codes; it does not validate that the response payload is well-formed.

Suggested fix — add a nil guard before the switch:

run := waited.Run
if run == nil {
    return 1, fmt.Errorf("job %q: Wait returned no run", name)
}
switch run.State {

Confidence	Score
🟡 moderate	67/100

[docker-agent]

[medium] createJobRun can return a nil *Run without an error, causing a nil dereference in the caller

Both return paths in createJobRun return reply.Run directly:

// manual job (line 492)
return reply.Run, nil

// scheduled job (line 503)
return reply.Run, nil

CreateAndRunReply.Run and RunReply.Run are both proto *Run pointer fields with omitempty, so they can be nil in a non-error reply. If createJobRun returns (nil, nil), the caller (RunJob) immediately dereferences created.ContainerID, created.JobID, and created.ID — all three are nil pointer dereferences.

Suggested fix — add a nil guard in createJobRun before returning:

if reply.Run == nil {
    return nil, fmt.Errorf("job %q: engine returned no run", name)
}
return reply.Run, nil

Apply the same guard to the RunRequest path as well.

Confidence	Score
🟡 moderate	67/100

[docker-agent]

[medium] Non-zero exit code from RunJob with a non-nil error silently discards the typed error

When backend.RunJob returns (exitCode != 0, err != nil), the typed err is flattened to a plain string in cli.StatusError.Status:

errMsg = err.Error()
return cli.StatusError{StatusCode: exitCode, Status: errMsg}

Any errors.Is/errors.As inspection downstream is broken — callers cannot detect errdefs.ErrNotFound, errdefs.ErrNotImplemented, or other typed errors that RunJob can return (via jobsv0.MapError). For example, if the engine is unreachable, the typed "not implemented" error is lost and the caller sees only a string.

The RunOneOffContainer path below uses the same pattern (pre-existing code, out of scope), but the RunJob path is new in this PR.

Consider adding the typed error to the struct or returning it directly when no meaningful exit code is available:

return cli.StatusError{StatusCode: exitCode, Status: err.Error(), Err: err}

(if cli.StatusError has an Err field — otherwise return the typed error directly when exitCode == 0 and wrap with the exit code otherwise).

Confidence	Score
🟡 moderate	70/100