Repository navigation
MLE-29419: Hash expression values in URI templates when writing documents - #688
Draft
prgs-rsingh wants to merge 1 commit into
Draft
prgs-rsingh wants to merge 1 commit into
prgs-rsingh wants to merge 1 commit into
Conversation
prgs-rsingh
requested review from
NiteshMehta,
adganta,
ashokrprogress,
hmadala001,
rjrudin and
vishwach
as code owners
October 13, 2026 09:25
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
End-to-end behavior depends on an open, currently blocked Spark connector pull request.
2 open findings
What changed in this PR
Adds SHA-256 hashing for URI-template expression values across Flux import and copy APIs.
Changes:
- Adds CLI and fluent API options for hashed URI-template values.
- Passes the connector option through import and copy workflows.
- Adds tests and usage/security documentation.
| File | Description |
|---|---|
flux-cli/src/main/java/com/marklogic/flux/api/WriteDocumentsOptions.java |
Adds the fluent hashing option. |
flux-cli/src/main/java/com/marklogic/flux/impl/importdata/WriteDocumentParams.java |
Implements import CLI/API option mapping. |
flux-cli/src/main/java/com/marklogic/flux/impl/copy/CopyCommand.java |
Implements copy CLI/API option mapping. |
flux-cli/src/test/java/com/marklogic/flux/impl/importdata/ImportFilesOptionsTest.java |
Tests import option mapping and default behavior. |
flux-cli/src/test/java/com/marklogic/flux/impl/copy/CopyOptionsTest.java |
Tests copy option mapping and default behavior. |
flux-cli/src/test/java/com/marklogic/flux/api/AggregateJsonFilesImporterTest.java |
Verifies generated hashed URIs. |
docs/import/common-import-features.md |
Documents import usage and pseudonymization. |
docs/import/incremental-write.md |
Documents incremental-write integration. |
docs/copy.md |
Documents the copy option. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| Similarly, to construct document URIs via a template, use `--output-uri-template`. As of Flux 2.2.0, you can use | ||
| `--output-uri-template-hash-values` to hash each resolved expression value in the template using SHA-256 (lowercase hex) | ||
| before constructing the document URI. You can also use `--output-uri-template-warn-on-missing-field` to log a warning and | ||
| replace missing expressions with a UUID instead of failing. |
|
|
||
| --uri-template "/employee/{id}/{last_name}.json" --uri-template-hash-values | ||
|
|
||
| When specified, every expression in the template is individually replaced by its 64-character lowercase hex digest. |
prgs-rsingh
marked this pull request as draft
October 13, 2026 11:20
rjrudin
reviewed
Oct 13, 2026
| * @return this options instance | ||
| * @since 2.2.0 | ||
| */ | ||
| T uriTemplateHashValues(); |
Contributor
There was a problem hiding this comment.
There's inconsistency here with the previous PR for inheriting permissions and collections, as to whether a method for a boolean value accepts an argument or not. I don't think it was 100% consistent before the 2.0 release. I would check all the methods in the API that map to boolean values and see what's most common, and then update either this PR or modify the previous feature to not use a boolean value. Generally, I like not having a boolean when it's a feature that's disabled by default.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements MLE-29419: Adds support for hashing expression values resolved from URI templates when writing documents in Flux import and copy commands.
Depends on: marklogic/marklogic-spark-connector#703
Key Changes
T uriTemplateHashValues()fluent builder method toWriteDocumentsOptionsAPI.--uri-template-hash-valuesCLI option toWriteDocumentParamsacross all import commands supporting--uri-template, passingOptions.WRITE_URI_TEMPLATE_HASH_VALUES = "true"to the Spark connector.--output-uri-template-hash-valuesCLI option anduriTemplateHashValues()method toCopyCommand.CopyWriteDocumentsParamsfor thecopycommand.docs/import/common-import-features.md,docs/import/incremental-write.md, anddocs/copy.mdwith usage instructions, examples, synergy with incremental writes, and the pseudonymization caveat.ImportFilesOptionsTest,CopyOptionsTest, andAggregateJsonFilesImporterTest.