Model key and trust store configurations as records and support PKCS#11

[beanuwave]

Description

Category: Enhancement, Refactoring

Independent of FIPS: this adds PKCS#11 (HSM) support for node TLS and replaces ad-hoc isPkcs11()-style branching with a type-driven model. Reviewable without any FIPS context.

Key changes

• Sealed records replace branching. Key stores and trust stores are modelled as pem / jdk / pkcs11 records, and PKCS#11 dispatch moves into the records themselves rather than being re-decided at each call site.
• StorePassword wrapper. Store passwords are wrapped in a type whose toString() redacts the value, so a password cannot reach a log through an accidental interpolation.
• A PKCS#11 store has no file. The key lives on the token, so the path becomes optional throughout: KeyStoreUtils loads such stores with a null stream, and error messages name the token instead of a non-existent file. PemKeyReader learns the PKCS11 store type and validates that a PKCS#11 provider is actually registered, rather than failing obscurely later.
• PKCS#11 keys are signed via SunJSSE, not BCJSSE. A PKCS#11/HSM private key is non-exportable and BCJSSE cannot sign with it (no encoding for key); SunJSSE delegates the handshake signature to the key's own provider (SunPKCS11). So when the keystore is PKCS#11, SslConfiguration builds the SSLContext against SunJSSE (SslContextBuilder.sslContextProvider(SunJSSE)).
• Ignored settings now warn. Trust store settings that a PKCS#11 configuration disregards produce a warning naming the exact setting, instead of being silently dropped.

Reviewer notes

• Reviewable as two commits: the records refactor (no capability change), then the PKCS#11 capability on top.
• One test placeholder: PemSslCertificatesLoaderTest uses nextAlphanumeric(10) where the final branch uses PasswordValidator.FIPS_MIN_PASSWORD_LENGTH. That constant does not exist until PR 5 — this is deliberate, please don't "fix" it here.

Testing

./gradlew test integrationTest

Manual test: node TLS key on a PKCS#11 token (SoftHSM), non-FIPS

Exercises a token-resident node TLS key (signed via SunJSSE) with a PKCS#12 file trust store, on a plain non-FIPS JVM. securityadmin still authenticates with the PEM kirk cert here — PKCS#11 client keys in the CLI arrive with PR 5. All paths are relative to $OPENSEARCH_HOME.

# 1. Set up a test cluster with the demo configuration.
cd $OPENSEARCH_HOME
sh plugins/opensearch-security/tools/install_demo_configuration.sh -y -i -s

# 2. Init token - the --pin becomes the keystore password.
softhsm2-util --init-token \
  --free \
  --label opensearch \
  --so-pin 4321 \
  --pin 1234

# 3. Configure SunPKCS11 in a java.security override (config/pkcs11.java.security).
#    Index 12 is the JDK's unconfigured SunPKCS11 slot; this line configures it.
      security.provider.12=SunPKCS11 /path/to/config/softhsm-pkcs11.cfg
#    with softhsm-pkcs11.cfg in the same dir:
      name = SoftHSM
      library = /usr/lib/softhsm/libsofthsm2.so
      slotListIndex = 0

# 4. Import the node key WITH its chain (PKCS#12 -> token).
openssl pkcs12 -export \
  -inkey config/esnode-key.pem \
  -in config/esnode.pem \
  -certfile config/root-ca.pem \
  -name esnode-cert \
  -out /tmp/esnode.p12 \
  -passout pass:1234

jdk/bin/keytool \
  -importkeystore \
  -srckeystore /tmp/esnode.p12 \
  -srcstoretype PKCS12 \
  -srcstorepass 1234 \
  -srcalias esnode-cert \
  -destkeystore NONE \
  -deststoretype PKCS11 \
  -deststorepass 1234 \
  -destalias esnode-cert \
  -addprovider SunPKCS11 \
  -providerarg config/softhsm-pkcs11.cfg

# 5. Trust anchor -> PKCS#12 file.
jdk/bin/keytool -importcert -noprompt \
  -alias root-ca \
  -file config/root-ca.pem \
  -keystore config/root-ca.p12 \
  -storetype PKCS12 \
  -storepass changeit

# 6. In opensearch.yml: comment the demo *.pem*_filepath lines, keep admin_dn: CN=kirk,...,
#    and add for both transport and http:

# --- Node identity from the PKCS#11 token (no keystore_filepath: the key lives on the token) ---
plugins.security.ssl.transport.keystore_type: PKCS11
plugins.security.ssl.transport.keystore_alias: esnode-cert
plugins.security.ssl.transport.keystore_password: "1234"      # SoftHSM PIN
plugins.security.ssl.http.keystore_type: PKCS11
plugins.security.ssl.http.keystore_alias: esnode-cert
plugins.security.ssl.http.keystore_password: "1234"

# --- Trust anchor from a PKCS#12 file ---
plugins.security.ssl.transport.truststore_type: PKCS12
plugins.security.ssl.transport.truststore_filepath: root-ca.p12
plugins.security.ssl.transport.truststore_password: "changeit"
plugins.security.ssl.http.truststore_type: PKCS12
plugins.security.ssl.http.truststore_filepath: root-ca.p12
plugins.security.ssl.http.truststore_password: "changeit"

# 7. Start the node with the override (single '=' appends to the JDK's java.security).
OPENSEARCH_JAVA_OPTS="-Djava.security.properties=$OPENSEARCH_HOME/config/pkcs11.java.security" \
  ./bin/opensearch

# 8. The HTTP layer serves the token-resident esnode cert (verify chain against root-ca):
openssl s_client -connect localhost:9200 -CAfile config/root-ca.pem </dev/null 2>/dev/null \
  | grep -E "subject=|Verify return code"          # expect CN=node-0.example.com, "0 (ok)"

# 9. Apply security config over that TLS layer. Pass = 'Done with success'.
sh plugins/opensearch-security/tools/securityadmin.sh \
  -cd config/opensearch-security/ \
  -icl \
  -nhnv \
  -cacert config/root-ca.pem \
  -cert config/kirk.pem \
  -key config/kirk-key.pem

Negative checks (restart per check, restore afterwards):

• Wrong PIN (keystore_password: "0000") — node refuses to start with Failed to load keystore from the PKCS#11 token.
• Start without OPENSEARCH_JAVA_OPTS (no PKCS#11 provider configured) — node refuses to start at config load with the same message, caused by KeyStoreException: PKCS11 not found, rather than failing later in a handshake.

Check List

• New functionality includes testing
• New functionality has been documented
• New Roles/Permissions have a corresponding security dashboards plugin PR
• API changes companion pull request created
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

PR Code Analyzer ❗

AI-powered 'Code-Diff-Analyzer' found issues on commit 6816422.

⛔ Hard block: Issues at High severity or above will block this PR from merging.

Path	Line	Severity	Description
build.gradle	606	high	New dependency org.bouncycastle:bctls-fips added (versions.bouncycastle_tls) across integrationTestImplementation, testImplementation, compileOnly, and bundled artifact scopes. Per mandatory rule, all dependency additions must be flagged for maintainer verification regardless of how legitimate the artifact name appears — namespace hijacking and typosquatting are common supply chain attack vectors.
src/test/resources/fips-jvm-truststore.bcfks	1	medium	A binary BCFKS trust store file is added to test resources and referenced as the JVM-wide trust store (-Djavax.net.ssl.trustStore) for all FIPS test workers. The certificates it contains cannot be audited from this diff. Any CA certificate inside it would be implicitly trusted during all FIPS test runs, including integration tests that make real TLS connections.
gradle/fips.gradle	69	low	The truststore password is hardcoded as the string literal 'changeit' and injected as a JVM system property (-Djavax.net.ssl.trustStorePassword=changeit) on every FIPS test worker. While 'changeit' is the conventional Java demo password and this is test-scoped, the plaintext credential is visible to all processes that can read JVM arguments.
src/main/java/org/opensearch/security/support/FipsMode.java	17	low	FipsMode.envSupplier is declared package-private and non-final, allowing any class in the same package to swap the supplier and silently disable FIPS mode detection at runtime. The test FipsModeTest exploits this deliberately; however, the same mutability exists in production code paths, meaning a class added to org.opensearch.security.support could suppress FIPS enforcement without touching FipsMode itself.

The table above displays the top 10 most important findings.

Total: 4 | Critical: 0 | High: 1 | Medium: 1 | Low: 2

---

Pull Requests Author(s): Please update your Pull Request according to the report above.

Repository Maintainer(s): You can bypass diff analyzer by adding label skip-diff-analyzer after reviewing the changes carefully, then re-run failed actions. To re-enable the analyzer, remove the label, then re-run all actions.

---

⚠️ Note: The Code-Diff-Analyzer helps protect against potentially harmful code patterns. Please ensure you have thoroughly reviewed the changes beforehand.

Thanks.