Repository navigation
0.2: owned Uri<V> BIP 321 parser with optional body address - #16
caarloshenriq wants to merge 8 commits into
Conversation
There was a problem hiding this comment.
I do see this built to the #12 checklist, but not exactly from the perspective of the actual problem. What is developed here needs to be demonstrated from the perspective of a consumer, like reading pj or sp with an example. Writing PjParam::from_uri would show that this change has no public decoder, so every consumer would have to undo the URI's percent-encoding itself. In 0.1 this crate decoded the value and payjoin only parsed it. Something like ParamEntry::decoded_value() would keep it that way.
Tests and review should be trying to break what was built instead of confirming, which is a similar symptom to #15. You can sic bots on this and they do very well. I want to know about your prompting and thinking in your validation section, rather than raw bot output.
Uppercase URIs survive for QR alphanumeric mode (regression test for #10 included).
check this for bech32. Display lowercases a bech32 body. BITCOIN:BC1Q…?AMOUNT=1 prints as BITCOIN:bc1q…?AMOUNT=1, and the #10 test uses a
base58 body, so it doesn't catch this.
both lockfiles regenerated
This repo doesn't commit lockfiles. If you mean rust-payjoin's in your #1867 branch depending on this, that's ok, but it has nothing to do with the design or review here. Please push that branch so we can see it, too. Doesn't need to be a PR but it's the sort of demo I talked about right in the beginning that makes reviewer verification possible.
no_std build and
cargo-mutantsare clean
cargo-mutants 27.1.0 with --all-features at acf7a69 finds 126 mutants: 4 missed, 94 caught, 28 unviable. The missed ones are both ParamEntry::is_required mutants and the Display impls for NetworkError and AccessError
Pushing back on #12's design would show more of an understanding of the problem. The design is the thing this PR process needs to work out. For example, a closed instruction list is against the spirit of the problem where payjoin and silentpayments, lightning crates (or something spun out of them, subcrates) do the parsing and provide it to this crate rather than have this crate do everything itself, like Mutiny's old bitcoin-waila crate. That closed list is in #12 because I put it there, and I'll update #12, my bad.
Then, I understand you tried to do small steps with PRs #13, #14, #15 without responses, so here's my suggestion for how to break this up, with each commit building and passing tests on its own:
- MSRV 1.85, bitcoin bump and CI. No behavior change.
- Internal Params/ParamEntry tokenizer plus the decoder, with its own tests (value-less params, percent validation, case-insensitive keys, base key). Nothing uses it yet.
- Switch Uri to the owned model and delete Extras/de/ser. This is the one big breaking commit, kept to the model swap. The body is still required.
- Make the body optional (bitcoin:?sp=…).
- require_network/is_valid_for_network over the body and bc/tb, plus the parse-time bc=tb1 rejection.
- unsatisfied_requirements(understood) plus the #7 test.
- Exact Display, source_str, and the #10 test with a bech32 body.
- UriBuilder.
CHANGELOG/README changes go in an ## Unreleased section, in the same commit as the change. See rust-payjoin's CONTRIBUTING.
get/get_all by base key. get("pj") also matches req-pj (the requirement check lives in unsatisfied_requirements). It reads cleanly to me, but it does mean the returned entry's .keycan be req-pj`. Easy to switch to exact-key matching if you prefer.
Yes this is the right ergonomics, req is just the prefix. The rest of the crate should follow the same pattern, see the inline comments on the HRP checks.
require_networkerror type. Network-mismatch is no longer a parse error, so I split it into its ownNetworkErrorrather than keeping it inError. Happy to fold it back if you'd rather keep one error type.
Keep it separate because Parsing never returns it and parsing is where Error is produced afaiu. build() now takes a network (see #12), so BuildError can wrap the same NetworkError
| //! **Important:** this is the 0.2 line, mid-rewrite against | ||
| //! <https://github.com/payjoin/bitcoin_uri/issues/12>. It is not published | ||
| //! and its API is not stable. 0.1 remains the published, stable crate. | ||
| //! |
There was a problem hiding this comment.
This type of comment does not make sense as module-level documentation. It's good feedback from a bot to you while you're working, but please make sure your bot is instructed to deliver that to you through some other medium, and not to me.
| if let Some(&(_, network)) = HRP_ADDRESS_KEYS.iter().find(|(hrp, _)| key.eq_ignore_ascii_case(hrp)) { | ||
| validate_hrp_param(key, raw_value.as_deref().unwrap_or(""), network)?; | ||
| } | ||
|
|
There was a problem hiding this comment.
req-bc=tb1… would bypass this conditional because it uses exact keys rather than checking for keys in the set of {{req-,}[HRP]} Make sure you can't bypass it.
| // A bc=/tb= value must be a valid address of its key's network - the | ||
| // same rule parsing enforces - so the built URI parses back. | ||
| for entry in &self.entries { | ||
| if let Some(&(hrp, network)) = HRP_ADDRESS_KEYS.iter().find(|(hrp, _)| entry.key_eq(hrp)) { |
There was a problem hiding this comment.
Same HRP bypass would let req- rpefixed keys get skipped.
| #[test] | ||
| fn a_req_prefixed_instruction_key_is_not_an_instruction() { | ||
| // `req-` handling belongs to requirement enumeration, not to listing | ||
| // what the URI offers. | ||
| let uri = Uri::parse("bitcoin:?req-sp=sp1silent") | ||
| .unwrap() | ||
| .require_network(bitcoin::Network::Bitcoin) | ||
| .unwrap(); | ||
| assert_eq!(instructions_of(&uri).len(), 0); | ||
| } |
There was a problem hiding this comment.
This test goes away with instructions() (see #12). Keep one that checks get("sp") finds req-sp.
| pub use builder::UriBuilder; | ||
| pub use error::{AccessError, BuildError, Error, NetworkError}; | ||
| pub use instruction::Instruction; | ||
| pub use params::{ParamEntry, Params}; |
There was a problem hiding this comment.
EITHER make builder, error, params private modules (drop pub) and keep the re-exports here OR keep pub mod and drop. Think it through and drop your rationale with your choice.
| .instruction("bc", BECH32_MAINNET) | ||
| .instruction("tb", BECH32_TESTNET) |
There was a problem hiding this comment.
Mixed networks should not be allowed in a "checked" URI.
|
Smaller things I didn't comment on inline:
|
acf7a69 to
e0210f5
Compare
|
Hey @DanGould, pushed: rewrote the branch into the 8 commits you suggested, each passing CI on its own. The crate is a pure tokenizer now. It parses the BIP 321 shape and hands back the raw params; interpreting pj/sp/lightning is the extension's job, not the crate's. I dropped The bugs you caught:
(CI aligned to 1.85, Errors stay separate: parse only produces Validation: on breaking it instead of confirming, I checked each commit with build, tests and Demo: I left builder/error/params private with re-exports at the root: the module layout stays an implementation detail and there's a single public path. Happy to make them |
e0210f5 to
65efdc6
Compare
|
Please make sure CI runs, could be just that the pins are wrong |
The crate declared no MSRV while CI tested 1.63.0, and the bitcoin dependency was unpinned within 0.32. Set rust-version to 1.85 to match rust-payjoin, pin bitcoin to 0.32.102, and update the CI test matrix to 1.85 so the declared MSRV is the one enforced.
Add Params/ParamEntry: an ordered, case-insensitive multimap of the raw query parameters that keeps every value percent-encoded and in source order, with ParamEntry::decoded_value for on-demand RFC 3986 decoding and AccessError for a value that fails to decode. Nothing consumes the tokenizer yet; the owned Uri wires it in next, so its crate-internal helpers are allowed dead for this commit.
Replace the borrowed Uri<'a, NetVal, Extras> with an owned Uri<V> that holds its body address and the Params tokenizer, and delete the de and ser serde-style modules and the Extras machinery. The body is still required; a later commit makes it optional. Drop the non-compliant-bytes feature and the either dependency: a non-UTF-8 value is now an access-time AccessError, not a separate parsing mode. Keep the error and params modules private and re-export their public types at the crate root, so the module layout stays an implementation detail.
BIP 321 allows a body-less URI such as bitcoin:?sp=..., where the payment rides an extension parameter instead of the on-chain address. Make the body an Option: parse accepts an empty body, Uri::body returns Option<&Address>, and Display emits the scheme and parameters with no address in between.
Add Uri::is_valid_for_network and Uri::require_network, which check the body address and every bc=/tb= parameter against one network and, on success, mark the URI checked. Reject a bc=/tb= value that is not a valid address for the network its key names at parse time, per BIP 321's own invalid example. The HRP check matches on the base key, so a req- prefix cannot smuggle a wrong-network address past it. Network mismatch is its own NetworkError, kept out of the parse Error since parsing never produces it.
Add Uri::unsatisfied_requirements(understood), which returns the req- parameters whose base key the caller did not list as understood. The crate reports the BIP 321 req- rule rather than enforcing it: a URI always parses, and the caller decides whether to reject it. The iterator is #[must_use] so the check is not dropped by accident. Add an extension_params example showing a consumer read pj, sp, and lightning off a parsed URI with get and decoded_value.
Keep the original string on a parsed Uri and re-emit the body verbatim from it. bech32 is case-sensitive, so an uppercase address for QR alphanumeric mode must survive the round trip rather than be lowercased by the Address Display. Add Uri::source_str for the original text, and exclude source from equality so a parsed URI and one built from the same parts compare equal. Regression test for issue payjoin#10 now uses a bech32 body, which the old base58 test could not catch.
Add UriBuilder, which assembles a Uri from an address, typed setters, and param/raw_param, and build(network) that validates the body and every bc=/tb= address against one network and rejects a mixed-network URI. A build-time failure is a BuildError, which wraps NetworkError for the mismatch case. Also round out unit-test coverage for the error Displays, the Params iterator, and the value decoder.
65efdc6 to
7cd6044
Compare
Ok, this last push was tested on act before push to upstream. |
|
Carlos, thanks for the restack. The series read well commit by commit, the crate is a pure tokenizer now, and your rust-payjoin branch was the demo I wanted. The I'm going to open a new PR with these commits plus the fixes below, and I'm closing this one in its favor so 0.2 can land. You're co-author on every commit. Here's what I found Review of 7cd6044
I also changed my mind on MSRV because bitcoin-payment-instructions supports 1.63 and this crate is meant to be a dependency of it, so 0.2 is 1.63. Master's 1.63 job had already been broken by an What kind of review would have caught theseYou said mutants is how you push back on the "just confirm" trap, and it did its job on the code that was there, but it's more of a long-term QA tool than a review tool. It can't see code that's missing. A key the crate doesn't know about, a check nobody wrote and bytes
|
|
Thanks Dan, this was a really useful review, especially the point about mutants not seeing code that isn't there. I've updated payjoin/rust-payjoin#1975 to depend on #17: |
|
Superseded by #17 ~ great work keeping this moving @caarloshenriq thank you for your contributions |
Closes #12. Unblocks payjoin/rust-payjoin#1867.
Rewrites the crate around an owned
Uri<V>so a URI is parsed once and every consumer reads the same structure, instead of each wallet running a second parser over the string (the source of the recurring+-as-space, lost-endpoint, and string-concatenation bugs).Data model
Uri<V: NetworkValidation>holds an optional bodyAddress<V>, a params multimap ofParamEntry { key, raw_value }that preserves raw percent-encoded values and original key case, and the exact input string. No lifetime parameter and noExtrastype parameter.Parsing
Values stay percent-encoded until accessed. Decoding is RFC 3986 only, so
+is always literal. Value-less parameters are allowed. Keys match case-insensitively;amount,label,message, andpopmay not repeat (by base key, soamountandreq-amountcollide), everything else may.Instructions
instructions()yields the body address and each parameter payload (bc=/tb=,lightning,lno,sp,pay) verbatim, in document order. The crate deliberately does not decode invoices, offers, or silent-payment keys, and staysno_std.Required parameters
The parser never rejects an unknown
req-key.unsatisfied_requirements(understood)returns the unhandledreq-entries and is#[must_use]; wallets treat a non-empty result as invalid per BIP 321.Builder
UriBuilderexposesaddress,amount,label,message,instruction(key, value),param(key, value), andraw_param(key, pre_encoded), with qchar-compliant encoding.build()rejects keys or raw values that would not round-trip, duplicate non-repeatable params, and a malformed raw amount, so builder output always re-parses to the sameinstructions and params.
Round-trip
Displayre-emits entries in original order, with original key case and encoded values, and preserves the original scheme case;source_str()returns the exact parsed input. Uppercase URIs survive for QR alphanumeric mode (regression test for #10 included).Network validation
require_networkandis_valid_for_networkcheck the body and thebc=/tb=params;assume_checkedis available.Errors
Parse-time:
InvalidScheme,InvalidBodyAddress,InvalidAmount,DuplicateParameter,InvalidPercentEncoding. Everything else, including non-UTF-8 values, surfaces at access time viaAccessError. All error types implementstd::error::Error.Breaking changes
Extrastype parameter, and therelated traits.
non-compliant-bytesfeature.bitcoinaligned with rust-payjoin.Validation
Each commit passes
cargo fmt --check,cargo clippy --all-targets --all-features -D warnings,RUSTDOCFLAGS=-D warnings cargo doc,cargo test --all-features, andcodespell; no_std build andcargo-mutantsare clean; both lockfiles regenerated. rust-payjoin's #1867 integration tests pass against this branch.Two design calls I'd like your read on
require_networkerror type. Network-mismatch is no longer a parse error, so I split it into its ownNetworkErrorrather than keeping it inError. Happy to fold it back if you'd rather keep one error type.get/get_allby base key.get("pj")also matchesreq-pj(the requirement check lives in unsatisfied_requirements). It reads cleanly to me, but it does mean the returned entry's.keycan bereq-pj`. Easy to switch to exact-key matching if you prefer.Follow-up
Fuzz targets for the parser to follow in a separate PR.
Disclosure: co-authored by Opus 4.8