[Storage] Detect CSE v2 region reorder

[Isabelle (ibrandes)]

Detect reordered authenticated regions in client-side encryption v2 (blob)

Summary

Fixes an edge case in client-side encryption (CSE) v2 integrity checking. Individual blocks of authenticated ciphertext (encryption "regions") could be reordered without detection -- the ciphertext was otherwise untampered and each region's GCM tag remained valid, so decryption silently produced corrupted plaintext. This is now detected during decryption and an exception is thrown.

Because CSE v2 is cross-SDK interoperable, the detection recognizes the per-region nonce encodings of all three SDKs that produce CSE v2 content (.NET, Java, Python), so it does not raise false positives on blobs written by another SDK. For data recovery, a compatibility switch can disable the check to allow the (potentially tampered) plaintext to be recovered.

Reason for the change

In CSE v2, blob content is encrypted in independent regions, each with its own nonce and GCM authentication tag. Because the nonce is stored inline with the ciphertext, an attacker (or corruption) can swap whole regions around; each region still authenticates in isolation, so the tampering goes undetected and decryption returns reordered plaintext.

Each SDK derives a region's nonce deterministically from its sequential region index. The fix validates, during decryption, that each region's nonce matches the value expected for its position under a single recognized nonce scheme, enforced consistently across the entire download. A mismatch throws.

How it works

• Each region's inline nonce is compared against the value expected for its sequential position. The starting region index is derived from the requested range (offset / regionDataLength), so full downloads, ranged downloads, downloadToFile, and openInputStream are all covered.
• The three known nonce encodings are modeled as schemes (Java: full-width big-endian counter in the leading 8 bytes; Python: full-width big-endian counter; .NET: 4 zero bytes + 8-byte little-endian, 1-based). The validator starts with all schemes and intersects the set still consistent with every region seen so far, collapsing to a single scheme as the download proceeds. This avoids false positives across SDKs while still catching a mid-blob scheme switch that only a reorder across the schemes' colliding value space could produce.
• A single validator instance is shared per logical download operation (via a pipeline-context key), so the scheme is enforced consistently across all chunks — including the concurrent ranged requests issued by a partitioned downloadToFile or a chunked openInputStream — rather than being re-established per chunk. The intersection is done atomically and is order-independent, so it is safe under concurrent region processing.
• Fails closed on an unsupported nonce length: CSE v2 nonces are always 12 bytes. If a region's advertised nonce length cannot hold the region index, positional validation is impossible, so decryption throws rather than silently skipping the check (still overridable by the recovery switch).

Changes

• CseV2NonceOrderValidator.java (new) — Core reorder-detection logic: the cross-SDK NonceScheme enum, the scheme-intersection state machine, the fail-closed guard for unsupported nonce lengths, and the live-read recovery switch. Reads the recovery switch directly from a system property/environment variable (not Configuration, which caches the first read and would ignore a value set later).
• DecryptorV2.java — Indexes each region during decryption and calls the shared validator with the region's inline nonce and its sequential index; the starting index is derived from the requested range. Now holds a CseV2NonceOrderValidator (falling back to a fresh per-call instance when none is supplied).
• Decryptor.java — getDecryptor threads the operation-scoped validator through to DecryptorV2.
• BlobDecryptionPolicy.java — Retrieves the operation-scoped validator from the pipeline context (both the full-blob and ranged branches) and passes it into decryptBlob/getDecryptor; falls back to a fresh per-call validator when none is present (e.g. a full-blob single-shot download).
• EncryptedBlobClient.java / EncryptedBlobAsyncClient.java — Install a fresh CseV2NonceOrderValidator into the download context alongside the encryption data so it is shared across every chunk of the operation.
• CryptographyConstants.java — Adds the context key (GCM_NONCE_VALIDATOR_KEY) and the recovery-switch property/env-var name constants, following the existing storage config/env-switch convention.
• CHANGELOG.md, .vscode/cspell.json — Changelog entry and dictionary addition (misordered).

Tests

• DecryptorV2ReorderTests.java (new) — Drives the real EncryptorV2/DecryptorV2 (no storage account required) and covers: unmodified content decrypts; reorder detected; reorder detected on a ranged download (correct region offset, no false positives); the recovery switch allowing reordered plaintext; fail-closed on an unsupported nonce length and the recovery-switch override for it; decryption and reorder detection across the Integer.MAX_VALUE region boundary; regions just below and at the 2³² boundary decrypt normally (full 64-bit counter); Python- and .NET-encoded blobs decrypt without a false reorder and their reorders are detected; a mid-blob scheme switch is rejected; the single-region ranged-download limitation is documented; and the shared validator enforces one scheme across chunks.
• EncryptedBlockBlobApiTests.java — Adds encryptionV2DetectRegionReorder (@LiveOnly, end-to-end: upload with CSE v2, swap two regions in the raw ciphertext, assert download now fails) and crossPlatDecryptPythonV2 (@Disabled, cross-SDK decryption of a Python-produced blob against a coordinated container), plus an A256KwKey test helper.
• DecryptionTests.java — Updated for the new decryptBlob(..., nonceValidator) signature (passes null).

Compatibility / data recovery

Detection is enabled by default. To recover data from an affected blob, set either:

• Environment variable AZURE_STORAGE_CSE_V2_ALLOW_MISORDERED_AUTH_REGIONS=true, or
• System property Azure.Storage.CseV2AllowMisorderedAuthRegions=true

When enabled, decryption does not throw and recovers plaintext in the received (reordered) order. The switch is read live per download operation, so it takes effect without requiring it to be set before the first CSE operation.

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).
34 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Pull request overview

Adds CSE v2 authenticated-region reorder detection during blob decryption.

Changes:

• Validates sequential region nonces with a recovery override.
• Adds unit and live coverage for reordered regions.
• Documents the behavior change.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
DecryptorV2.java	Implements nonce-order validation and override handling.
CryptographyConstants.java	Defines override configuration names.
DecryptorV2ReorderTests.java	Tests detection, ranges, and compatibility mode.
EncryptedBlockBlobApiTests.java	Adds live end-to-end reorder coverage.
CHANGELOG.md	Documents the fix and recovery override.

---

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

sdk/storage/azure-storage-blob-cryptography/src/main/java/com/azure/storage/blob/specialized/cryptography/DecryptorV2.java:134

• This exact comparison rejects untampered CSE v2 blobs produced by other official SDKs. Java writes an 8-byte big-endian int counter followed by four zero bytes, but Python writes the counter as a 12-byte big-endian value, while the linked .NET implementation validates a 1-based little-endian counter in the final eight bytes. These blobs previously decrypted because the inline nonce was passed directly to GCM; now region 1 already mismatches unless callers disable the integrity check. Preserve cross-SDK downloads by recognizing a consistent supported producer encoding and add .NET/Python ciphertext fixtures.

        byte[] expectedNonce = ByteBuffer.allocate(nonceLength).putLong((int) expectedRegion).array();
        if (Arrays.equals(expectedNonce, actualNonce)) {

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Suppressed comments (1)

sdk/storage/azure-storage-blob-cryptography/src/main/java/com/azure/storage/blob/specialized/cryptography/DecryptorV2.java:159

• Fail closed when the nonce length cannot be validated. Returning null here silently disables the new integrity check for malformed CSE v2 metadata even though detection is enabled; CSE v2 uses a 12-byte nonce, so an unsupported short nonce should be rejected unless the explicit recovery switch has already bypassed validation.

        // Cannot reconstruct the expected nonce if it is too short to hold the region index. This should never happen
        // for CSEv2 (nonce length is 12), so treat it as unverifiable rather than a failure.
        if (nonceLength < Long.BYTES || actualNonce.length < Long.BYTES) {
            return null;
        }

[Copilot]

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).
34 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Jocelyn (jaschrep-msft)]

Some cleanup needed but looks good from my end otherwise

[Copilot]

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.

Suppressed comments (2)

sdk/storage/azure-storage-blob-cryptography/src/main/java/com/azure/storage/blob/specialized/cryptography/CseV2NonceOrderValidator.java:154

• This only recognizes the post-#50205 full-width Java encoding. Blobs written by earlier Java releases used (int) regionIndex and diverge at region 2^31 (EncryptorV2NonceTests.java:153-156); because 16-byte regions are supported, that can occur around 32 GiB. Such valid existing blobs will now be rejected as reordered. Please retain a legacy-Java candidate through its non-repeating range, fail it closed at 2^32, and keep the full-width candidate for newly written blobs, with compatibility tests for both.

         * Java: the region index written as an 8-byte big-endian value in the first 8 bytes, remaining bytes zero.
         * 0-based. See {@link EncryptorV2}.
         */
        JAVA {

sdk/storage/azure-storage-blob-cryptography/src/test/java/com/azure/storage/blob/specialized/cryptography/DecryptorV2ReorderTests.java:319

• This JavaDoc contradicts the helper: javaNonce now encodes the full long and the boundary tests rely on that behavior. Update it so future tests do not mistake this helper for the legacy truncated encoding.

     * Encrypts a single region using the Java SDK nonce scheme (region index truncated to an int, written big-endian).

[Copilot]

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated no new comments.

Suppressed comments (3)

Previously missed (3) — in code that hasn't changed since the last review.

sdk/storage/azure-storage-blob-cryptography/src/main/java/com/azure/storage/blob/specialized/cryptography/EncryptedBlobClient.java:502

• This security-critical context wiring is not exercised by the added tests. sharedValidatorEnforcesSchemeAcrossChunks injects the same validator directly, while the live reorder test uses a full downloadStream, which bypasses populateRequestConditionsAndContext; therefore, removing this context entry or accidentally creating one validator per chunk would leave all tests green. Add a client/policy test that performs two ranged chunks in one sync operation (for example, downloadToFile or openInputStream), uses one nonce scheme in the first chunk and another in the second, and verifies that the second chunk is rejected.

                .addData(GCM_NONCE_VALIDATOR_KEY, new CseV2NonceOrderValidator())

sdk/storage/azure-storage-blob-cryptography/src/main/java/com/azure/storage/blob/specialized/cryptography/EncryptedBlobAsyncClient.java:768

• The async operation-scoped sharing path has no automated coverage. The direct decryptor test manually supplies a shared validator, and the only new client-level reorder test is synchronous and does not execute this contextWrite, so a regression that gives each async download chunk a fresh validator would not be detected. Add an async downloadToFile/policy test with two ranges that establish one scheme in the first range and switch schemes in the second, then assert that the shared validator rejects it.

                    .contextWrite(context -> context.put(GCM_NONCE_VALIDATOR_KEY, new CseV2NonceOrderValidator()))

sdk/storage/azure-storage-blob-cryptography/CHANGELOG.md:10

• This release note says the fix applies only to encryption 2.0, but Decryptor.getDecryptor applies the validator to both ENCRYPTION_PROTOCOL_V2 and ENCRYPTION_PROTOCOL_V2_1, and the new live test uses V2_1. Calling it “v2” avoids incorrectly implying that 2.1 users are unaffected.

- Fixed a bug where client-side encryption 2.0 could not detect a rearrangement of otherwise-untampered authenticated regions in blob content. This is now detected and an exception is thrown. For data recovery purposes, this behavior can be reverted by setting the `AZURE_STORAGE_CSE_V2_ALLOW_MISORDERED_AUTH_REGIONS` environment variable (or the `Azure.Storage.CseV2AllowMisorderedAuthRegions` system property) to `true`.

[browndav-msft]

This might be a bit cleaner and a little more dry, if his were a static method on CseV2NonceOrderValidator e.g. CseV2NonceOrderValidator.fromContext(context)

[browndav-msft]

https://github.com/Azure/azure-sdk-for-java/pull/50121/changes#diff-7bfe76684b229a93b61cfda697e2bce02158e50cade7b5b669203cee8b3a20f2R141