sync: add --concurrency flag

[igrekun]

Changes

Expose the previously-hardcoded request concurrency cap as a --concurrency flag on databricks sync and databricks bundle sync. Defaults to the existing limit (20) so behavior is unchanged when the flag is omitted; values < 1 are rejected.

The flag name matches databricks fs cp --concurrency (#4132).

Why

When uploading multiple files ~5mb size, high concurrency can trigger stream timeouts.

Tests

[github-actions]

Approval status: pending

/acceptance/bundle/ - needs approval

Files: acceptance/bundle/help/bundle-sync/output.txt
Suggested: @pietern
Also eligible: @denik, @andrewnester, @janniklasrose, @shreyas-goenka, @anton-107, @lennartkats-db

/cmd/bundle/ - needs approval

Files: cmd/bundle/sync.go
Suggested: @pietern
Also eligible: @denik, @andrewnester, @janniklasrose, @shreyas-goenka, @anton-107, @lennartkats-db

/cmd/sync/ - needs approval

Files: cmd/sync/sync.go, cmd/sync/sync_test.go
Suggested: @simonfaltum
Also eligible: @mihaimitrea-db, @renaudhartert-db, @hectorcast-db, @parthban-db, @tanmay-db, @Divyansh-db, @tejaskochar-db, @chrisst, @rauchy

/libs/sync/ - needs approval

Files: libs/sync/sync.go, libs/sync/watchdog.go
Suggested: @simonfaltum
Also eligible: @mihaimitrea-db, @renaudhartert-db, @hectorcast-db, @parthban-db, @tanmay-db, @Divyansh-db, @tejaskochar-db, @chrisst, @rauchy

General files (require maintainer)

Files: NEXT_CHANGELOG.md, acceptance/cmd/sync-without-args/output.txt
Based on git history:

• @pietern -- recent work in libs/sync/, ./, cmd/sync/

_{Any maintainer (@andrewnester, @anton-107, @denik, @pietern, @shreyas-goenka, @simonfaltum, @renaudhartert-db) can approve all areas.
See OWNERS for ownership rules.}

[pietern]

Thanks for the PR!

Small suggestion on naming the flag and vars.

[pietern]

There is precedent for configuring concurrency in fs cp in #4132.

Please mirror the flag name, handling, etc, for consistency.

[pietern]

Can be Concurrency.

[pietern]

@renaudhartert-db @simonfaltum FYI

[pietern]

@igrekun I applied the suggestions to the PR, please take a look and confirm this matches what you need.

Also includes the same flag to bundle sync to keep it consistent.

[simonfaltum]

My concern is the validation order for databricks sync.

Right now PreRunE runs root.MustWorkspaceClient before RunE validates --concurrency, so invalid flag values can get masked by auth/config errors. I verified this with:

go run . --profile __missing_profile_for_review__ sync --concurrency 0 /tmp /Workspace/tmp

This returns the missing-profile error instead of --concurrency must be at least 1.

I think we should move f.validate() into PreRunE, before mustWorkspaceClient, similar to how fs cp validates its concurrency flag before constructing the workspace client. That makes the CLI reject invalid user input before doing auth setup.