[SPARK-57897][BUILD] Create connector/credential-aws module with CI/release integration

[sarutak]

What changes were proposed in this pull request?

This PR creates the connector/credential-aws module scaffolding as a part of the OIDC Credential Propagation SPIP (SPARK-57703).

Module definition:

• connector/credential-aws/pom.xml (parent: spark-parent_2.13, deps: spark-core provided, spark-tags, junit-jupiter test)
• Root pom.xml: <profile><id>credential-aws</id> with <module>connector/credential-aws</module>
• project/SparkBuild.scala: added credentialAws to optionallyEnabledProjects and MiMa exclusion list
• dev/sparktestsupport/modules.py: credential_aws Module definition
• dev/sparktestsupport/utils.py: doctest expected output update

ServiceLoader registration and stub provider:

• META-INF/services/org.apache.spark.security.CredentialProvider pointing to AwsStsCredentialProvider
• Stub AwsStsCredentialProvider — minimal implementation for ServiceLoader discovery testing. This class will be replaced by the full implementation in SPARK-57898 (Sub-task 9). The stub is included here to verify that the ServiceLoader wiring, test framework setup, and CI profile activation work end-to-end.

CI/release/lint script updates (-Pcredential-aws added in 22 locations):

• .github/workflows/build_and_test.yml (is-changed.py module list, SBT build, Maven build)
• .github/workflows/maven_test.yml (all build/test commands)
• .github/workflows/python_hosted_runner_test.yml
• .github/workflows/benchmark.yml
• dev/create-release/release-build.sh
• dev/test-dependencies.sh
• dev/mima
• dev/lint-java
• dev/sbt-checkstyle
• dev/scalastyle
• docs/_plugins/build_api_docs.rb
• dev/spark-test-image-util/docs/build-docs

Note: This PR establishes the build infrastructure only (stub provider). The actual provider implementation and its tests will follow in SPARK-57898, which will exercise the profile end-to-end (compilation, test execution, lint checks).

Why are the changes needed?

The AWS reference provider needs a separate module to keep AWS SDK dependencies out of Spark core. Splitting the module scaffolding from the implementation:

1. Makes the CI/release integration independently reviewable (12 scripts, 22 locations).
2. Allows SPARK-57898 and SPARK-57899 to focus purely on the provider implementation and tests after rebasing onto this PR.

Does this PR introduce any user-facing change?

No. The module is activated only via -Pcredential-aws (Maven) or by including credential-aws in SBT's optionallyEnabledProjects. It is not included in the default build. The stub provider throws UnsupportedOperationException on init() and resolve().

How was this patch tested?

• mvn -Pcredential-aws -pl connector/credential-aws test: 5 tests pass (ServiceLoader discovery, supportedSchemes, init throws, resolve throws, suggestedTtl)
• build/sbt -Pcredential-aws "credential-aws/test": 5 tests pass
• python3 -m doctest dev/sparktestsupport/utils.py: passes with updated expected output
• ./dev/lint-java: Checkstyle checks passed
• ./dev/mima: success
• ./dev/scalastyle: Scalastyle checks passed
• mvn -Pcredential-aws -pl connector/credential-aws apache-rat:check: all source files have valid license headers

Was this patch authored or co-authored using generative AI tooling?

Kiro CLI / Claude