Repository navigation
0.2: owned Uri<V> BIP 321 parser with optional body address - #17
Conversation
| @@ -3,6 +3,7 @@ name = "bitcoin_uri" | |||
| version = "0.1.0" | |||
| authors = ["Dan Gould <d@ngould.dev>"] | |||
| edition = "2018" | |||
There was a problem hiding this comment.
we might as well bump this to edition = "2021" since we are bumping the rust-version already to improve some navigability here.
|
Windows CI seems broken. As well I wonder if the Fuzz commit should just be rolled in to replace the existing uri targets over in the monorepo https://github.com/payjoin/rust-payjoin/tree/master/fuzz/fuzz_targets/uri. |
Agree on moving the fuzz targets into the monorepo. The existing uri targets there only check that parsing doesn't panic, so build_parse and parse_display would be a real upgrade, and they'd run with the rest of the fuzzing setup instead of on their own here |
caarloshenriq
left a comment
There was a problem hiding this comment.
Went commit by commit and tried to break the claims rather than just read it, per your last comment.
What held up:
- fmt, clippy
-D clippy::alland tests pass on every commit. - Both fuzz targets ran clean here: about 14M runs of parse_display and 2M of build_parse.
- The edges the targets don't generate behave the same in parse and build: signet with
tb=, upper-caseBC=BC1Q..., taproot underbc=, and empty or value-lessamountandbc(rejected on both sides). - Nothing in the graph
cargo testactually builds needs more than 1.63 onceccis pinned. - rust-payjoin#1975 builds against 3bc80f9 with
network_ofgone
The crate declared no MSRV while CI tested 1.63.0, and that job could not have passed since a lint pass brought back an is_some_and in the scheme check (stable since 1.70). Restore map_or, declare rust-version = "1.63", and require bitcoin 0.32.102 or later. 1.63 matches bitcoin-payment-instructions, which this crate is meant to be a dependency of. rust-payjoin at 1.85 can depend on a crate with a lower MSRV. The 1.63 CI job pins cc to 1.1.31 first, because later cc releases pull in find-msvc-tools, which needs 1.65. The actions move to dtolnay/rust-toolchain and actions/checkout@v6. Edition moves to 2021 in the same pass, while the manifest is being touched. Windows leaves the test matrix. This crate moves into the rust-payjoin monorepo after 0.2, which already checks Windows at release time only, and the windows-latest 1.63 job wedged the GitHub runner for six hours compiling secp256k1-sys before the timeout canceled it. Co-authored-by: Carlos Santos <rickhenriquetab@gmail.com>
Add Params/ParamEntry: an ordered, case-insensitive multimap of the raw
query parameters that keeps every value percent-encoded and in source
order, with ParamEntry::decoded_value for on-demand RFC 3986 decoding
and AccessError for a value that fails to decode.
Lookups compare base keys. req- is only a prefix, so it is stripped from
the stored key and from the key asked for: get("pj") and get("req-pj")
both find pj and req-pj.
Nothing consumes the tokenizer yet; the owned Uri wires it in next, so
its crate-internal helpers are allowed dead for this commit.
Co-authored-by: Carlos Santos <rickhenriquetab@gmail.com>
Replace the borrowed Uri<'a, NetVal, Extras> with an owned Uri<V> that holds its body address and the Params tokenizer, and delete the de and ser serde-style modules and the Extras machinery. The body is still required; a later commit makes it optional. Drop the non-compliant-bytes feature and the either dependency: a non-UTF-8 value is now an access-time AccessError, not a separate parsing mode. Keep the error and params modules private and re-export their public types at the crate root, so the module layout stays an implementation detail. Co-authored-by: Carlos Santos <rickhenriquetab@gmail.com>
BIP 321 allows a body-less URI such as bitcoin:?sp=..., where the payment instruction is in an extension parameter instead of the on-chain address. Make the body an Option: parse accepts an empty body, Uri::body returns Option<&Address>, and Display emits the scheme and parameters with no address in between. Co-authored-by: Carlos Santos <rickhenriquetab@gmail.com>
Add Uri::is_valid_for_network and Uri::require_network, which check the body address and every bc=, tb= and bcrt= parameter against one network and, on success, mark the URI checked. Network mismatch is its own NetworkError, kept out of the parse Error since parsing never produces it. At parse time a value under an HRP key must be a segwit address whose HRP is the key, or the parse fails with Error::InvalidHrpValue. That covers BIP 321's own invalid example, bitcoin:?bc=tb1..., and also a base58 address, which require_network alone lets through because it only knows mainnet or test. bcrt is a key from the start: adding a key later would change which URIs parse. The check matches on the base key, so req-bc=tb1... is rejected too. Co-authored-by: Carlos Santos <rickhenriquetab@gmail.com>
Add Uri::unsatisfied_requirements(understood), which returns the req- parameters whose base key the caller did not list as understood. The crate reports the BIP 321 req- rule rather than enforcing it: a URI with an unknown req- parameter still parses, and the caller decides whether to reject it. The iterator is #[must_use] so the check is not dropped by accident. Add an extension_params example showing a consumer read pj, sp, and lightning off a parsed URI with get and decoded_value. Co-authored-by: Carlos Santos <rickhenriquetab@gmail.com>
Keep the original string on a parsed Uri and write it back verbatim. A
Uri cannot be changed after parsing, so the source is always the URI it
describes, and parse then display gives back the input: an uppercase
bech32 body for QR alphanumeric mode, scheme case, empty parameters, a
trailing ? or & and a #fragment all survive. Rebuilding the string from
the parsed parts would drop the last four, and the Address Display would
lowercase the body.
A Uri with no source, and the alternate {:#} form, are written from the
parts as bitcoin:<body>?<params>.
Add Uri::source_str for the original text, and exclude source from
equality so a parsed URI and one built from the same parts compare
equal. The issue Kixunil#10 regression test now uses a bech32 body, which the
old base58 test could not catch.
Co-authored-by: Carlos Santos <rickhenriquetab@gmail.com>
Add UriBuilder, which assembles a Uri from an address, typed setters, and param/raw_param. Its output parses back to the same Uri. build() takes no network. A checked Address does not say which network it belongs to, so a caller would have to guess one. Instead each bc=, tb= and bcrt= value is checked against the network its key names, with the same validator the parser uses, and the body against that network. Two HRP keys naming different networks are a mismatch, so a built URI never mixes networks. A URI with no HRP-keyed parameter takes the body as given. Keys and raw values must be BIP 321 qchars, RFC 3986 query characters other than & and =, with valid % escapes. Values set through the typed setters and param are percent-encoded to qchars for the caller. A build-time failure is a BuildError, which wraps NetworkError for the mismatch case. Also round out unit-test coverage for the error Displays, the Params iterator, and the value decoder. Co-authored-by: Carlos Santos <rickhenriquetab@gmail.com>
Copy every URI from BIP 321's appendix into an integration test. The valid ones parse, pass require_network and display as written. The invalid ones fail with the error each describes. The proof-of-payment callbacks decode to the URIs the BIP says a wallet opens, and the req- examples are reported by unsatisfied_requirements. The examples' body address, 175tWpb8K1S7NmH4Zx6rewF9WQrcZv245W, comes from BIP 21 and has a bad base58 checksum, so the tests swap in a valid mainnet address and assert that the original is rejected. Co-authored-by: Carlos Santos <rickhenriquetab@gmail.com>
3bc80f9 to
680704b
Compare
| /// The `req-` parameters this URI requires but the caller does not | ||
| /// understand, in source order. | ||
| /// | ||
| /// BIP 321 says a client MUST reject a URI carrying a `req-` parameter it | ||
| /// cannot handle. `understood` is the set of keys the caller does handle, | ||
| /// given without the `req-` prefix (`["pj", "sp"]`, not `["req-pj"]`) and | ||
| /// matched case-insensitively; every `req-` parameter whose stripped key | ||
| /// is absent from it is returned. An empty iterator means the URI is safe | ||
| /// to act on. | ||
| /// | ||
| /// This crate never enforces the rule itself - it only reports it - so a | ||
| /// `Uri` always parses regardless of its `req-` parameters, and the | ||
| /// caller decides what to do with any this returns. |
There was a problem hiding this comment.
Following up on my earlier note: this still reports req-amount, req-label, req-message and req-bc unless the caller lists them, even though the crate parses those itself. The extension_params example passes ["pj", "sp", "lightning"], so anyone copying it rejects ?req-amount=1. Could the doc (or the example) say the typed keys need to be in understood too? Not blocking.
|
|
||
| let mut params = Params::new(); | ||
| if let Some(query) = query { | ||
| // RFC 3986 §3.4: the query ends at the first '#', if any. |
There was a problem hiding this comment.
I think this comment is relevant enough to include in the doc string itself for Uri::parse
| if: matrix.rust == '1.63' | ||
| run: cargo update -p cc --precise 1.1.31 | ||
| - name: Test | ||
| run: cargo test |
There was a problem hiding this comment.
We might want to include --all-features as we don't default to including std as a default feature.
| let (key, raw_value) = match segment.find('=') { | ||
| // `otherparam = qchar *qchar [ "=" *qchar ]`: the value | ||
| // is optional, so a value-less param is legal. | ||
| Some(pos) => (&segment[..pos], Some(segment[(pos + 1)..].to_owned())), | ||
| None => (segment, None), | ||
| }; | ||
|
|
||
| if let Some(raw) = raw_value.as_deref() { | ||
| validate_percent_encoding(key, raw)?; | ||
| } | ||
|
|
||
| let base_key = params::base_key(key); | ||
| if params::NON_REPEATABLE_KEYS | ||
| .iter() | ||
| .any(|&non_repeatable| base_key.eq_ignore_ascii_case(non_repeatable)) | ||
| && params.get(base_key).is_some() | ||
| { | ||
| return Err(Error::DuplicateParameter(key.to_owned())); | ||
| } |
There was a problem hiding this comment.
RFC 3986 makes percent encoding equivalent to the non-reserved characters so %61mount is equivalent to amount but gets around Error::DuplicateParameter. I wonder if this is something we should be trying to solve for event though this falls squarely in non-rust-payjoin territory as this would require some broken uri generation. https://datatracker.ietf.org/doc/html/rfc3986#section-2.3
There was a problem hiding this comment.
I think it's proper to treat amount and %61mount as distinct. BIP[3]21's amountparam is the literal "amount=" and %61mount= doesn't match so it's an otherparam not something to trigger DuplicateParameter. Likewise %72eq-foo is an unknown optional key, not a req- parameter. As you say this is only something we'd be dealing with from broken URI gen, so let's follow the grammar literally and leave normalization out of scope. The builder can reject % in keys as a follow-up so we never emit one ourselves.
Address #17 follow-ups to prep release 0.2.0
Closes #12. Supersedes #16. Unblocks payjoin/rust-payjoin#1867.
This is @caarloshenriq's #16 with review fixes folded into his commits, plus BIP 321 vector tests and fuzz targets. He is co-author on every commit.
Rewrites the crate around an owned
Uri<V>that is parsed once, so a wallet reads one structure instead of running a second parser over the string. A second parser is where the recurring+-as-space, lost-endpoint and string-concatenation bugs came from.Data model
Uri<V: NetworkValidation>holds an optional bodyAddress<V>, the parameters as an ordered multimap ofParamEntry { key, raw_value }, and the string it was parsed from. Values stay percent-encoded and keys keep their case. There is no lifetime parameter and noExtrastype parameter. An extension such aspj,sporlightningreads its value withget/get_alland decodes it withParamEntry::decoded_value. This crate doesn't need to know what the value means.Parsing
Decoding is RFC 3986 only, so
+is always a literal plus. Value-less parameters are allowed. Keys match case-insensitively, and lookups compare base keys, soget("pj")andget("req-pj")both findpjandreq-pj.amount,label,messageandpopmay not repeat by base key, sopopandreq-popcollide. Any other key may repeat.bc,tbandbcrttake only a segwit address whose HRP is the key.bitcoin:?bc=tb1...andbitcoin:?bc=1...both fail withError::InvalidHrpValue.Required parameters
The parser never rejects an unknown
req-key.unsatisfied_requirements(understood)returns thereq-entries the caller doesn't handle and is#[must_use]. A wallet treats a non-empty result as an invalid URI, as BIP 321 requires.Network validation
require_networkandis_valid_for_networkcheck the body and everybc/tb/bcrtparameter. A mismatch is aNetworkError, separate from the parseError.assume_checkedis available too.Display
Displaywrites a parsed URI back exactly as it was parsed, including an uppercase QR body, a trailing?or&, and a#fragment.source_str()returns the same string. A built URI is written asbitcoin:<body>?<params>, and{:#}uppercases the body.Builder
UriBuilderhasaddress,amount,label,message,param(key, value)andraw_param(key, encoded).build()takes no network. It checks eachbc/tb/bcrtvalue against the network its key names, using the parser's check, and then checks the body against that network. A URI that mixes networks fails withBuildError::Network. Keys and raw values must be BIP 321qchars. Builder output always parses back to the sameUri.Errors
At parse time:
InvalidScheme,InvalidBodyAddress,InvalidHrpValue,InvalidAmount,DuplicateParameterandInvalidPercentEncoding. A value whose decoded bytes aren't UTF-8 is anAccessErrorwhen it is read. All error types implementstd::error::Errorwith thestdfeature.Breaking changes
Extrastype parameter and the serde-style traits are gone.non-compliant-bytesfeature is gone.bitcoinmust be 0.32.102 or later. rust-payjoin at 1.85 can depend on it.Disclosure: co-authored by Claude Code