diff --git a/_typos.toml b/_typos.toml new file mode 100644 index 0000000..ff49751 --- /dev/null +++ b/_typos.toml @@ -0,0 +1,3 @@ +[default.extend-words] +# A given name, in a citation. +Damon = "Damon" diff --git a/cspell.json b/cspell.json index c90a361..949b01a 100644 --- a/cspell.json +++ b/cspell.json @@ -21,7 +21,8 @@ "python", "networking-terms", "bitcoin-privacy", - "repo-tooling" + "repo-tooling", + "background" ], "dictionaryDefinitions": [ { @@ -31,6 +32,10 @@ { "name": "repo-tooling", "path": "./cspell/tooling.txt" + }, + { + "name": "background", + "path": "./cspell/background.txt" } ] } diff --git a/cspell/background.txt b/cspell/background.txt new file mode 100644 index 0000000..40d81e4 --- /dev/null +++ b/cspell/background.txt @@ -0,0 +1,88 @@ +# Authors, projects and terms cited in docs/background, and misspellings quoted from sources +Aloni +Androulaki +Arvind +beliece +Bernasconi +bitcoinj +Böhme +Boneh +Calvez +Capkun +CIOH +Claessens +clusterable +Coinswap +Damiano +Danezis +deanonymized +Deidentification +Dorit +Elli +emember +entropist +Fergal +fingerprintable +Friedhelm +Ghassan +Goldfeder +Gruber +Haaroon +Hamacher +Haslhofer +Ishaana +Kalodner +Kappos +Karame +Katzenbeisser +Katzenpost +keyspend +Kirill +Kogman +Kristov +Latanya +Levchenko +Loporchio +Maesa +Maffei +Marjori +Maurer +Meiklejohn +Misra +Mixmaster +mixnet +mixnets +Monero +Möser +multisigs +Nordholt +nothingmuch +Pomarole +Preneel +Rainer +Reisman +Ricci +Roeschlin +Rollet +Romiti +Samourai +Serjantov +Seys +Srdjan +Stefaan +strawmen +Stütz +Syverson +textrm +Tramer +unlinkable +unredacted +Voelker +walletexplorer +Weili +xpubs +Yakun +Yousaf +Yuval +Zcash +Zhenghao diff --git a/docs/background/faces-of-coinjoin.md b/docs/background/faces-of-coinjoin.md new file mode 100644 index 0000000..ab158fb --- /dev/null +++ b/docs/background/faces-of-coinjoin.md @@ -0,0 +1,107 @@ +--- +type: Essay +title: "The many faces of CoinJoins" +description: "What \"CoinJoin\" refers to: equal amount CoinJoins, multiparty batching, PayJoin and Lightning." +status: stable +--- + +# The Many Faces of CoinJoins + +> Originally published as [The Scroll #5: The Many Faces of CoinJoins](https://spiralxyz.substack.com/p/the-scroll-5-the-many-faces-of-coinjoins) by Yuval Kogman +> ([@not_nothingmuch](https://x.com/not_nothingmuch)) on Spiral, +> 2025-10-29. + +## What Does “CoinJoin” Refer To? + +What even is a “CoinJoin?” The long answer is: “it depends.” The tl;dr is: it means any kind of multiparty transaction, which technically makes Lightning a kind of CoinJoin. ;-) + +### Equal Amount CoinJoins + +In 2013, Greg Maxwell posted the following in the Bitcoin Forum: “[CoinJoin: Bitcoin privacy for the real world](https://bitcointalk.org/index.php?topic=279249.0).” Although this wasn’t the first account of breaking the common input ownership heuristic using theft-resistant multiparty transactions[^1], as far as I know it’s the first concrete description of multiparty transactions where the output values are the same[^2]: + +> *To use this to increase privacy, the N users would agree on a uniform output size and provide inputs amounting to at least that size. The transaction would have N outputs of that size and potentially N more change outputs if some of the users provided input in excess of the target. All would sign the transaction, and then the transaction could be transmitted. No risk of theft at any point.* + +A diagram was also included, and although the original URL is now broken a copy still exists on the [related Bitcoin wiki article](https://en.bitcoin.it/wiki/CoinJoin): + +
+ + +
+ +Note that the address 1A1 is used twice, strongly implying that the same entity owns the first and last inputs of Transaction 2. The post continues: + +> In the illustration ‘transaction 2’ has inputs from 1A1 and 1C3. Say we beliece \[sic\] 1A1 is an address used for Alice and 1C3 is an address used for Charlie. Which of Alice and Charlie owns which of the 1D and 1E outputs? + +The symmetry of this transaction structure makes every permutation of the same valued outputs equally plausible, at least when looking only at the transaction. If a single transaction is seen as a “data release” in the sense of the literature on k-anonymity, then assuming all of the attributes (the amount and the script type) are the same for k of the outputs, then this satisfies the definition of k-anonymity. Unfortunately, as discussed in the [previous post](intersection-attacks.md), this does not adequately account for privacy leaks from multiple transactions, how they are related. + +k-anonymity might be a reasonable model for CoinJoin privacy in principle, if the entire blockchain were seen as a continually appended data release. Unfortunately because transactions are connected in a graph structure the attribute set must also describe how every coin relates to every other coin in this graph, e.g. is one an ancestor of the other? This is a potentially quadratic growth in the number of attributes, but much worse, the number of possible quasi identifiers grows exponentially in the number of attributes. It is impossible to make rigorous claims about the k-anonymity of a particular coin meeting some threshold without accounting for the context in which the coin was created and the context in which it has been or will be spent. Showing that k is greater than some threshold, even with this proliferation of relevant quasi-identifiers, is possible in some specific transaction graph structures. Still, in general, this theory is intractable for any meaningful threshold. + +### Multiparty Batching and the Sub-Transaction Model + +Maxwell’s post is explicitly more general than just this highly symmetric approach. Another example in the post was that of collaborating to make batch transactions by aggregating several independent payments, or sub-transactions: + +> The idea can also be used more casually. When you want to make a payment, find someone else who also wants to make a payment and make a joint payment together. Doing so doesn’t increase privacy much, but it actually makes your transaction smaller and thus easier on the network (and lower in fees); the extra privacy is a perk. + +This was implemented more or less literally in SharedCoin, which indeed [didn’t increase privacy much](https://www.coindesk.com/markets/2014/06/10/blockchains-sharedcoin-users-can-be-identified-says-security-expert), as [demonstrated by Kristov Atlas](https://www.coinjoinsudoku.com/). The basic idea behind this analysis was that even if the inputs and outputs of a transaction were shuffled, related inputs and outputs can often still be uniquely [partitioned](https://en.wikipedia.org/wiki/Partition_of_a_set) just by assuming that the input and output values in each sub-transaction must be more or less balanced. The common input ownership heuristic can be seen as a degenerate special case of this theory, where the partitioning only contains one sub-transaction, implying that everything is linked to everything else. + +LaurentMT used a probabilistic approach in [Boltzmann](https://medium.com/@laurentmt/introducing-boltzmann-85930984a159), which accounted for when there was more than one unique partition satisfying the balance requirement. By enumerating and counting the different possibilities, and then normalizing these counts to a probability, this makes an estimate of how likely it is that inputs or outputs might be linked to one another. Sub-transactions are allowed to be only approximately balanced, considering things like JoinMarket’s maker fees. + +Later work by [Maurer et al](https://sci-hub.se/https://ieeexplore.ieee.org/abstract/document/8029483) introduced a double counting correction, distinguishing between so-called derived and non-derived sub-transaction mappings (their term for the partitioning as a whole), and only counting the latter. This work neglects to account for fees entirely, working in a more idealized setting. + +Although the total number of [partitions](https://en.wikipedia.org/wiki/Partition_of_a_set) of a set of size n—which is given by the nth [Bell number](https://en.wikipedia.org/wiki/Bell_number)—grows very rapidly, the subset of those partitions that satisfy this condition that the amounts are balanced may be very sparse within it. It is tempting, but incorrect to conclude from the rapid growth of the space that the complexity of finding some sparse set of solutions in such a vast search space is enough to protect privacy. + +The reason this misconception is tempting is that the closely related problem of subset sum is NP-complete, which means that it’s possible to construct a specific subset sum instance where finding a solution is not just as difficult as, but [equivalent to](https://en.wikipedia.org/wiki/Karp%27s_21_NP-complete_problems), just as an example, finding a SHA256 preimage for a specific hash. However, what NP-completeness does not guarantee is that all or even most subset sum problem instances are hard. This is why the [Merkle–Hellman knapsack cryptosystem](https://en.wikipedia.org/wiki/Merkle%E2%80%93Hellman_knapsack_cryptosystem) was broken, and other [Knapsack cryptosystems](https://en.wikipedia.org/wiki/Knapsack_cryptosystems) rely on hard special cases of subset sum. Linear programming is often practical for enumerating sub-transaction mappings of bitcoin transactions, even those of considerable size. + +What can provide a meaningful assurance is knowing that solving for *the* sub-transaction mapping is underdetermined, i.e. that there is more than some threshold of seemingly valid, non-derived sub-transaction mapping, or more precisely, that every input or output co-occurs with many other inputs and outputs in a varied collection of equally plausible but dissimilar sub-transactions across the range of sub-transaction mappings. + +### PayJoin + +In a [PayJoin](https://payjoin.org/) transaction, which the Bitcoin Wiki [describes](https://en.bitcoin.it/wiki/PayJoin) as a “special type of CoinJoin,” the sender and receiver of a payment both contribute inputs. + +Although the following excerpt from Maxwell’s post is overly optimistic in relation to multiparty batching, PayJoins attempts to realize the described indistinguishability property: + +> Such a transaction is externally *indistinguishable* from a transaction created through conventional use. Because of this, if these transactions become widespread they improve the privacy even of people who do not use them, because no longer will input co-joining be strong evidence of common control. + +For this to hold, the so-called “unnecessary input heuristics”\[[1](https://gist.github.com/AdamISZ/4551b947789d3216bacfcb7af25e029e), [2](https://eprint.iacr.org/2022/589.pdf)\] must also hold within the transaction. For example a transaction that spends two coins, say a 10 and a 4 to create two outputs worth an 8 and a 6, that raises the question of why it shouldn’t just use the 10 to create either an 8 and a 2, or a 6 and a 4, presuming either 8 or 6 is the intended payment amount. Although there are exceptions, such a transaction is somewhat unusual considering the distribution of transactions most wallets would construct, which would typically avoid unnecessarily using the second input as that would result in higher fees. + +This type of analysis can be extended beyond just the values. As briefly mentioned in the [post before last](history-of-wallet-clustering.md), wallet fingerprinting is not as well studied in the context of wallet clustering as perhaps it should be. Although it remains an open question, it appears likely that this can be used to attack PayJoin’s privacy, but it’s also possible to mitigate this concern. This is something we’re looking into as part of [PayJoin Dev Kit](https://payjoin.org), so if you’re interested in this kind of thing, please [join our Discord](https://discord.gg/6rJD9R684h)! + +Another way of describing this steganographic property of PayJoins is that interpreting these transactions using the sub-transaction model will almost certainly assign both the sender and the receiver’s inputs and outputs to the same sub-transaction. This is because a significant net transfer between them upsets the balance invariant, the definition of a sub-transaction mapping case excludes what actually happened. + +Breaking the sub-transaction invariant aids privacy even if we assume a hypothetical PayJoin aggregation mechanism that allows multiple senders and receivers to collaboratively construct a single transaction, something that would certainly not be distributed like “conventional use”. Although such multiparty PayJoin transactions could no longer be described as steganographic, seeing as they stand out, the missing information about the payments between participants makes clustering significantly harder. The more interconnected this activity, the more naive application of the theory of sub-transactions would lead to cluster collapse. + +### Seriously, Though, Lightning is a Kind of CoinJoin + +In the opening paragraph I framed it as a joke, and although I’ve [previously](history-of-wallet-clustering.md) mentioned it, let’s consider this angle a bit more seriously and fully here, because it does emphasize some aspects of Lightning that aren’t very widely discussed, and which aren’t as obvious when using a more sensible mental model to think about how Lightning works. + +A Lightning channel is a multisig UTXO owned by two parties sharing a channel. The transaction that creates it is its funding transaction. Some channels are funded by only one party, but some are dual-funded, meaning both parties contributed to the channel’s initial balance. The Lightning protocol ensures that at every point in time, either party to the channel can close it unilaterally (without depending on the counterparty’s cooperation). Closing means the UTXO gets spent into a transaction that fairly distributes the channel funds between them, according to a balance they both agreed to. Payment channels are valuable because this balance can be securely updated if both parties agree, and this update happens off-chain, without needing to spend the shared UTXO, which means that it’s a cheap and instant way of transferring bitcoin. Lightning channels in particular are even more valuable, because they also allow payments to be routed: channels are edges of a graph, linking together pairs of nodes, and payments can flow along a path of edges, between nodes that aren’t directly connected by sharing a channel. + +If everything goes smoothly, channels are closed cooperatively. In this scenario the channel UTXO is just spent into a transaction that settles the balance by creating two outputs. Unilateral closes also do this, but reveal more information and have a different on-chain footprint. We’ll ignore that here. Let’s also assume for a moment that the adversary we’re concerned with has no visibility into what happens off-chain, all routed payments have perfect privacy against this adversary. If all this adversary can see is the transactions that eventually confirm on-chain, then it would see a funding transaction that spends one or more coins to create a multisig coin (and maybe some change outputs). Then this multisig coin will be spent, creating two outputs, likely different amounts than what went into the funding transaction, each belonging to a different wallet. If it weren’t for the intermediate multisig coin, that all would sound a bit like a “conventional” payment transaction, or if the channel was dual funded, a PayJoin. + +In reality any adversary would be able to collect Lightning gossip data as well. For payments to be routed, nodes must have a view of the channel graph, which is what this gossip data makes possible. Publicly announced channels overtly and publicly link a pair of nodes. Although the clustering of multisigs is more nuanced than we’ve previously discussed, you might say that a Lightning node’s ID being linked to its channel (U)TXOs is a form of address reuse, and it wouldn’t be stretching the truth by that much. For the two outputs of a channel closure, there are two clusters related to the node IDs. Deciding which of the two clusters to assign each output to is relatively easy, since other channel openings and closures would be linked to each node, as well as any the preceding or succeeding transactions to which each cluster might extend. + +In other words, if we insist on thinking of Lightning as a strange form of CoinJoin or PayJoin, then its privacy guarantees are not very strong from a wallet clustering perspective. Please do not misread the previous sentence, it says nothing about the privacy of Lightning payments, which is by all accounts better than that of on-chain payments. Nor should the last sentence imply Lightning privacy is perfect. Anyway, the specific concern highlighted by thinking of Lightning channels as a weird 2 party CoinJoin is only to do with the on-chain footprint related to node opening and closing public channels, and the degree to which those funds can be clustered due to the non channel TXOs. This is also a good example of when clustering relevant information that isn’t recorded directly in the blockchain is still publicly available. + +### Accounting for Any Theory of Relatedness + +Generalizing further, within each transaction we might consider the inputs and outputs being connected in a weighted, [complete bipartite](https://en.wikipedia.org/wiki/Complete_bipartite_graph) graph, or even a complete graph (i.e. also allowing input-input links and output-output links to be represented). The weights on the edges indicate how related coins are. Indeed, there is no reason to stop at the boundary of a single transaction: all coins may in principle be related to all other coins, a probabilistic generalization of wallet clustering and inter-transaction relatedness, not just of the sub-transaction model and intra-transaction relatedness. + +The specific values assigned as these weights could be based on any other theory, such as the intersection attacks [discussed in the previous post](intersection-attacks.md#intersection-attacks), external data sources like [leaked xpubs](https://x.com/SamouraiWallet/status/1271155297495781376)[^3], KYC information or proprietary clustering labels, or even something as trivial as address reuse. + +And it can also go the other way. For example if it was widely accepted as true that transactions using [CoinSwap](https://github.com/citadel-tech/CoinSwap)[^4] are being created, even if it isn’t actually true, then anyone who knew this was widely believed and honestly attempts to assign weights in the linking graph would be compelled to account for that contingency and admit some degree of uncertainty, or argue to the contrary while bearing the burden of proof. An estimate of the upper limit of potential Coinswap usage for a more limited design was made in a [2017 paper](https://informationsecurity.uibk.ac.at/pdfs/MB2017_AnonymousAlone.pdf) by Malte Möser and Rainer Böhme. Chris Belcher’s [CoinSwap design](https://gist.github.com/chris-belcher/9144bd57a91c194e332fb5ca371d0964) compels the analysis to cast a much wider net. That said, it has not yet been fully implemented. Hopefully P2TR MuSig2 and multiparty ECDSA support will be added at some point, because even if those script types don’t get much use, the fungibility of coins of all types will improve as a result. The more it is true that for all anyone receiving a coin knows, it could have potentially originated from a CoinSwap (even if that’s not very likely), the weaker the theories of chain analysis become. Consider a scenario where a website breach leaks home addresses and bitcoin payments, and a large UTXO was used for one of these payments. A world in which it’s plausible that this UTXO belonged to a CoinSwap maker would be safer for this unfortunate customer, even if they never did a CoinSwap themselves. How much safer depends on how widely held a belief that is. + +This extends even to nonsensical, dare I say religious theories of input and output relatedness which have been proposed, such as [FIFO](https://archive.is/e7FsM)[^5] or ordinal theory. The unfortunate consequence of meme magic is that such theories can in some sense become good theories merely by [inhabiting people’s minds](https://en.wikipedia.org/wiki/Thomas_theorem) so that they end up shaping people’s actions, so that the trail of transactions that will be left behind can after the fact be correctly interpreted using the theory. Much like you shouldn’t take seriously anyone who tells you, without qualifying, that bitcoin is private, you also shouldn’t believe anyone who tells you bitcoin can’t be private. The outcome primarily depends on whether people who use bitcoin for commerce also use bitcoin in a privacy preserving manner. It may be unfortunate that this isn’t easy to do, at least with the current state of most wallets, especially not without deliberate effort, but I feel like the most embarrassing way to destroy Bitcoin privacy is if everyone “knows” it’s a lost cause because of some silly superstition. + +### On the Next Episode… + +In this post we’ve looked at the more theoretical side of CoinJoin privacy as it relates to different notions of the term, and then strayed a bit further than the CoinJoin sandwich alignment chart to see how these ideas can help make sense of Bitcoin privacy more broadly. The theory tends to emphasize the attackers’ perspective, since it’s often useful to reason about how hard it is to break privacy by understanding the attacker’s capabilities and finding the weaknesses. Although there is a lot more that could be said in theory, the next post will focus more on the defenders’ side, and discuss some of the practical challenges of designing a privacy preserving CoinJoin transaction structure. + +[^1]: [This comment](https://bitcointalk.org/index.php?topic=12751.msg315793#msg315793) is the earliest description I know of that idea. + +[^2]: That said, [this comment](https://bitcointalk.org/index.php?topic=12751.msg232155#msg232155) earlier in the same thread does mention non-uniformity of output values, so it seems likely the synthesis of the two ideas was made earlier than Maxwell’s post. + +[^3]: “If they only ever used dojo” is how this should have been phrased if they were honest, but even then it would only be truthful in a narrow, technical sense. Since the whirlpool protocol is vulnerable to trivial and non-trivial tagging attacks, and both the Samourai run server and the newly relaunched one control which inputs share a transaction, there is much reason for concern for the CoinJoin privacy of dojo and non-dojo users alike. + +[^4]: Note that at the time of writing mainnet use is still not recommended. + +[^5]: This is [published academic work](https://www.cl.cam.ac.uk/archive/rja14/Papers/making-bitcoin-legal.pdf) which seems to seriously suggest on the basis of common-law that wallet programs are somehow constrained in how they order transaction outputs. It rationalizes this absurd claim with a quasi-empirical use of the word “precise” that I can only describe as rhetorical sleight of hand. If that were true, this would mean all other published work on change identification heuristics over the years was pointless. And indeed, the authors pretty much say that outright: “People who have been doing research on financial anonymity without paying attention to \[the common law precedent for FIFO\] have simply been using the wrong metric.” diff --git a/docs/background/headaches-of-coinjoin.md b/docs/background/headaches-of-coinjoin.md new file mode 100644 index 0000000..26d2e4d --- /dev/null +++ b/docs/background/headaches-of-coinjoin.md @@ -0,0 +1,164 @@ +--- +type: Essay +title: "The many headaches of CoinJoin" +description: "Why designing a CoinJoin that overtly optimizes for privacy is harder than the folk anonymity set model suggests." +status: stable +--- + +# The Many Headaches of CoinJoin + +> Originally published as [The Many Headaches of CoinJoin](https://spiralxyz.substack.com/p/the-many-headaches-of-coinjoin) by Yuval Kogman +> ([@not_nothingmuch](https://x.com/not_nothingmuch)) on Spiral, +> 2026-09-03. + +CoinJoin has [many faces](faces-of-coinjoin.md), and it also has many headaches. This post will specifically focus on the cluster headache, pun unapologetically intended, that is designing an overtly privacy-optimizing CoinJoin structure. + +## Anonymity sets packed with strawmen + +The “folk” notion of an equal-amount CoinJoin anonymity set, based on the analogy to mixnets, unfortunately rests on some fairly strong and often tacit assumptions. These assumptions limit the model’s applicability for understanding the capabilities of real-world adversaries. + +In this conception of on-chain privacy, a CoinJoin transaction is analogized to a single shuffling stage in a mixnet. In each such transaction, there are *k* equivalent outputs that are presumed to be shuffled. Because they are equivalent, in the absence of auxiliary information not on the blockchain, there is ostensibly nothing to link any one output to any particular cluster associated with the inputs. + +A real-world adversary can incorporate later spending patterns, clustering techniques, and auxiliary information. It is not constrained to only examine the CoinJoin, with no memory of other transactions. + +If you want to skip the rest of this post, the main takeaway is that these outputs almost inevitably become more distinguishable in a variety of ways. The discussion is not exhaustive. Sometimes these privacy leaks will be enough for an adversary to link post-CoinJoin coins to their pre-CoinJoin ancestors successfully, but it’s hard to say to what degree. + +## The adversary doesn’t necessarily have tunnel vision + +The most obvious tacit assumption of the folk anonymity set model is that the CoinJoin structure can be considered separately from the rest of the transaction graph. + +Real-world adversaries of bitcoin users might be abusive spouses, criminals looking to steal or extort (sometimes aided by government officials), dragnet surveillance by governments, advertising companies, each with their different purposes in mind. A bitcoin user is exposed to harm from such interests if they lose deniability or leak information about their spending habits, net worth, security posture, etc. Depending on how motivated such a real-world adversary is, they may spend more resources attempting to surveil their victims. Still, it’s not reasonable to assume such adversaries can’t access a block explorer. + +It is customary to think of the adversary as an algorithm trying to output its best guess about the links. If this algorithm is constrained, and given only a single transaction in isolation, and no memory, although that might be enough to express some of the capabilities of the real-world adversaries, it already fails to capture wallet clustering, which is literally the oldest trick in the book (i.e., in the bitcoin privacy literature). + +A related misconception is that for some transactions breaking privacy is computationally intractable, which makes them private. Privacy should not rely on non-standard computational hardness assumptions (and relying on standard ones is just a fancy way of saying “using cryptography”). + +## Auxiliary information, metadata privacy + +Another tacit assumption is that it is enough to consider just the data on-chain. Unfortunately, external blockchain auxiliary information is clearly useful for deanonymization. + +Such information might be confidential compliance records (e.g. AML/KYC data available exchanges, or companies to whom they outsource identity verification) or other personally identifying information (e.g. email or physical addresses leaked in a website breach), temporal fingerprints (e.g. periods of activity indicating a likely time zone, but also things like latency), IP addresses, and implementation specific behavior that might reveal a user’s choice of wallet. + +Although this is important to consider, for this post we’ll now assume transport-level privacy works as a black box. + +A much stronger assumption we will leave intact is that it is easy to design a safety margin that accounts for any other auxiliary information related to the on-chain structure. The next post in the series will not entirely remove that assumption, but can at least provide a framework for treating it as an engineering problem as opposed to a tribal identity. + +We will return to the subject of auxiliary information in the last section of this post. + +## Sybil resistance + +The *n*-1 or Sybil deanonymization attack gives the adversary information about who the user is *not*. If all users but one are under adversarial control, all of their inputs and outputs can be excluded. This leaves the remaining honest user with effectively no anonymity, but maintains the appearance of an intact anonymity set. + +This challenge is inherent in a permissionless system. Such “[Sybil” users](https://en.wikipedia.org/wiki/Sybil_attack), many pseudonymous identities of a single entity, are indistinguishable from honest users. Therefore, Sybil resistance can only come from imposing some cost on participation. Some costs are inherent in bitcoin: transaction fees and time value of money. Additional costs can be imposed through mechanism design, but e.g. in the case of coordination fees, note that those decrease the cost of a Sybil attack by a corrupted coordinator, but that is largely a theoretical concern because thus far every centrally coordinated protocol on bitcoin has required trust in the coordinator at the protocol level. + +For bitcoin transactions specifically, the number of honest users may be large without much impact if the honest users’ actions don’t support each other’s privacy. For example, if the values of their coins are too far apart. + +For now, we will assume Sybil resistance can be taken for granted, i.e., that some mechanism enforces that such an attack is not costless, as that is the only kind of mitigation possible in the permissionless setting. Note that whatever mechanism is used, it shouldn’t degrade the privacy of the honest users. Where this is relevant to the on-chain structure, it will come up again, but otherwise Sybil resistance will be out of scope for this post. + +## Other users must also protect their privacy + +Even if we assume other users are not under adversarial control, if they lose their privacy due to carelessness or any other reason, the net effect is essentially the same as if they colluded with the adversary. Unlike a Sybil attack, sometimes it at least might be obvious that one of the other participants has lost privacy entirely. + +
+When the CoinJoin is created, from Alice’s perspective it’s indistinguishable from the best case privacy: each equal output appears equally attributable to any participant. +When the CoinJoin is created, from Alice’s perspective it’s indistinguishable from the best case privacy: each equal output appears equally attributable to any participant. +
+ +When the CoinJoin is created, from Alice’s perspective it’s indistinguishable from the best case privacy: each equal output appears equally attributable to any participant. + +A trivial example is reusing a public key used before a CoinJoin in a subsequent transaction, overtly linking a particular input and output. If Alice indexed all known addresses on the blockchain, she would be able to notice this, but that is costly. Needless to say, for the adversary, such costs are negligible and do not present a meaningful barrier. From the attacker’s point of view, it looks more like this at the time the CoinJoin is broadcast: + +
+The adversary can recognize that Bob’s output reuses a public key from before the CoinJoin. Note that this doesn’t require the adversary also to know that Bob’s input is Bob’s in this example. However, that is what is depicted; merely knowing the output belongs to Bob eliminates that output from being owned by Alice from the adversary’s point of view. The remaining candidate set shrinks from four participants to three. +The adversary can recognize that Bob’s output reuses a public key from before the CoinJoin. Note that this doesn’t require the adversary also to know that Bob’s input is Bob’s in this example. However, that is what is depicted; merely knowing the output belongs to Bob eliminates that output from being owned by Alice from the adversary’s point of view. The remaining candidate set shrinks from four participants to three. +
+ +The adversary can recognize that Bob’s output reuses a public key from before the CoinJoin. Note that this doesn’t require the adversary also to know that Bob’s input is Bob’s in this example. However, that is what is depicted; merely knowing the output belongs to Bob eliminates that output from being owned by Alice from the adversary’s point of view. The remaining candidate set shrinks from four participants to three. + +A more subtle example is linking through clustering heuristics. Such loss might be as damaging as address reuse, but it would typically be harder to notice, especially if it results from multiple data points rather than one unambiguous privacy error. + +
+In this example, Carol spends her CoinJoin output together with an input of hers that the adversary knows about. The common input ownership heuristic (see the first post in the series) identifies her output, leaving only Alice and Dave indistinguishable. +In this example, Carol spends her CoinJoin output together with an input of hers that the adversary knows about. The common input ownership heuristic (see the first post in the series) identifies her output, leaving only Alice and Dave indistinguishable. +
+ +In this example, Carol spends her CoinJoin output together with an input of hers that the adversary knows about. The common input ownership heuristic (see the [first post](wallet-clustering-basics.md) in the series) identifies her output, leaving only Alice and Dave indistinguishable. + +In general, such privacy losses may not even be apparent on-chain, e.g., another user disclosing KYC information to an exchange. This can even happen after the fact, and the damage to the other users’ privacy will be retroactive. Perhaps the most egregious such deanonymization occurred when, as part of litigation, Celsius users’ transaction history was inadequately “anonymized” (see [the post before last](intersection-attacks.md) for a discussion about anonymization failures), further harming not only direct victims of the fraud, but any parties they may have interacted with. + +
+If the adversary has auxiliary information identifying Dave, then Alice’s output is deterministically linkable to her input, undoing all anonymity. +If the adversary has auxiliary information identifying Dave, then Alice’s output is deterministically linkable to her input, undoing all anonymity. +
+ +*If the adversary has auxiliary information identifying Dave, then Alice’s output is deterministically linkable to her input, undoing all anonymity.* + +Setting this class of hazards aside, we will assume other users are both competent and vigilant in their pursuit of privacy. + +## Change outputs are toxic for privacy + +On a fundamental level, predicting payment amounts and future fee amounts has an element of market speculation. Even in cases when these might be reasonably predictable, the likelihood that a CoinJoin denomination would correspond to the required amount without producing change is low. + +Suppose Alice has two coins with value *d* from two independent CoinJoin transactions, that *d* is the smallest denomination in common use, and that there were *k* such values in each of the CoinJoins. She then makes two independent payment transactions, each for an amount smaller than *d*. At this point the anonymity sets of the payment transactions are indeed as the naive analysis suggests. + +
+Two CoinJoin output coins spent independently: each payment is ambiguous, and nothing overtly links the payments to the same user on-chain. +Two CoinJoin output coins spent independently: each payment is ambiguous, and nothing overtly links the payments to the same user on-chain. +
+ +*Two CoinJoin output coins spent independently: each payment is ambiguous, and nothing overtly links the payments to the same user on-chain.* + +The value of each change output depends on the nature of Alice’s economic activity, as do the fees, so unless these are somehow fully determined well in advance (and denominated in bitcoin), she will need to deal with change outputs whose amounts will vary. This is more or less unavoidable in the UTXO model. + +If Alice attempts to recover the value of these coins by spending them separately, each subsequent transaction will be linked to its predecessor. Worse, if the two coins are spent together, that transaction, by being linked to both prior payments, also undoes their privacy. + +Next, suppose that the sum of the two change outputs happens to be *d* or more. This would allow Alice to CoinJoin again and recover the value in these change outputs. Unfortunately, with current CoinJoin implementations, this too will undermine privacy. + +
+Even if the change outputs are only spent in CoinJoin transactions, the privacy of payment transactions and that of the user's wallet more broadly are likely to become undermined as a result of attempting to improve them. +Even if the change outputs are only spent in CoinJoin transactions, the privacy of payment transactions and that of the user's wallet more broadly are likely to become undermined as a result of attempting to improve them. +
+ +*Even if the change outputs are only spent in CoinJoin transactions, the privacy of payment transactions and that of the user's wallet more broadly are likely to become undermined as a result of attempting to improve them.* + +Even if the change outputs are only spent into CoinJoin transactions, the payment transactions’ privacy and that of the user’s wallet more broadly is likely to become undermined as a result of attempting to improve it. + +Fortunately, as we will see, this is not an inherent problem in CoinJoin transactions in the looser sense of the term, but it is inherent if only mix-style transactions are used. + +## Coin selection + +The previous section argued that change creation is inevitable. If only a single input is used for each transaction, the value of such a change output will necessarily be smaller than that of the input. If a wallet is subject to this constraint, over time its balance will get increasingly fragmented to the point of uselessness. For this reason, consolidation by spending more than one input per transaction is also inevitable. The more severe the fragmentation, the greater the degree of consolidation. + +This inevitability provides some intuition for why CIOH is as effective as it is, but a less intuitive threat is that of intersection attacks. As [previously discussed](intersection-attacks.md), intersection attacks are much more powerful. While CIOH is only concerned with what is implied by a single transaction, assuming the adversary can examine the transaction graph to arbitrary depth and intersect coin histories, common points of origin may be identified much more easily than is often assumed. Especially if the targets for intersection are clusters obtained from auxiliary information, not just those based on local heuristics. + +Even under the very generous assumptions we’ve made, we can see that CoinJoin transactions still have some irreducible friction that must be accounted for. + +## Auxiliary information (example scenario) + +If we now let go of these simplifying assumptions and return to auxiliary information, as we’ve already seen, it can take many forms. With the additional knowledge of what must be considered on-chain, we can now work through an example scenario in a bit more detail, and examine how such information might be used in the real world. + +Say Alice is paid monthly from her employer Emily’s cold storage wallet. This takes the form of a large, batch transaction with outputs for various employees. If we assume all employees are paid the same, then outwardly this transaction might resemble a CoinJoin, but at the very least Emily knows which output belongs to which employee, even as the employees only know which output is theirs and 3rd party observers know even less. + +Suppose that in subsequent batch payments, Emily’s change output from the previous transaction would sometimes be used to fund the next transaction. This loosely links the wallet clusters of Emily and her employees from the perspective of 3rd party observers. Emily’s coins would be in the history intersection of all of these clusters. The transactions themselves will be fairly identifiable if carried out periodically, especially if most of the output amounts imply a fixed fiat amount. + +Suppose Alice CoinJoins every one of her outputs after being paid by Emily. Even though her pay is unlikely to correspond to any particular denomination useful for CoinJoining, since that was already discussed in the previous section for the following example, assume this is not an issue. Note the strength of these assumptions; this is more or less the best-case scenario for using mix-style CoinJoins, and even here there are difficulties. In reality, all of these sources of friction give rise to privacy leaks which will compound (as opposed to merely accumulating, which would still be bad). + +Next, Alice would like to make a large purchase. Perhaps now that she’s saved up some coins, she needs a boat to secure her savings. Can she do this privately? If she treats her CoinJoin outputs as “anonymous” and consolidates them into a single payment, the answer is most likely “no”. + +For example, Bob the boat salesman will likely learn information about the origin of Alice’s income by intersections, because Emily’s coins are predecessors of all of Alice’s coins. The more coins Alice consolidates, the more these patterns will become apparent and unambiguous from Bob’s perspective. + +Similarly, Alice’s coworker Carol can also see that one of her colleagues likely bought something expensive. Carol has slightly more power to deanonymize Alice than Bob, because the structure of Emily’s transactions is clear, and she can further eliminate herself as a target, reducing some of Bob’s uncertainty. + +For similar reasons, Emily can also infer that Alice made an expensive purchase. Although Alice CoinJoined the coins she received directly from Emily, the logic of an intersection attack still works with respect to the set of Alice’s coins given that Emily knows the members of this set. This is trivially true if Alice is the only employee using CoinJoin, but even if the other employees are CoinJoining, even if they mostly participate in the same transactions, the power of an intersection attack will likely allow Emily to tell the employees apart and infer that it’s Alice who made the large purchase. + +Only if some other employee participated in every CoinJoin transaction that Alice made prior to the purchase will she have plausible deniability from Emily[^1]. Finally, if Emily is herself a boating accident aficionado, happened to transact with Bob, and he in turn was not careful to avoid linking his received coins, Emily will further learn that Alice’s transaction was with Bob. + +While this scenario was a bit contrived to illustrate how perspectives and privileged information may vary, in reality similar and more powerful capabilities are available to counterparties and 3rd parties. + +## Conclusion + +For real-world transaction graphs, subject to the constraints of the UTXO model, the naive anonymity set size estimate based on counting equivalent sibling outputs can be misleading and provide a false sense of privacy, even under fairly generous assumptions. This is even more true when the adversary has access to higher-quality auxiliary information, and when attacking the privacy of multiple users, not just one specific target. + +In the next post, we’ll explore some privacy failure modes across a range of structures, from PayJoin to a CoinJoin structure that takes an alternative approach to equal output amounts. For claims about privacy to be credible, any proposed solution must, of course, account for the state of the art (and not so state of the art but so far unaddressed) deanonymization attacks described in the literature, so the next post will try to define necessary, and hopefully sufficient, conditions for on chain transactions to be considered private. + +[^1]: If Emily's cold storage setup involves multisig these coins be more fingerprintable too, unless it's done with e.g. MuSig or FROST in a Taproot keyspend path. Even Multiparty ECDSA, while potentially blending into P2{,W}PKH set is unlikely to match the distribution of signature sizes, and so might be distinguishable from her employee's wallet fingerprints. diff --git a/docs/background/history-of-wallet-clustering.md b/docs/background/history-of-wallet-clustering.md new file mode 100644 index 0000000..c4df009 --- /dev/null +++ b/docs/background/history-of-wallet-clustering.md @@ -0,0 +1,90 @@ +--- +type: Essay +title: "A brief history of wallet clustering" +description: "A survey of the academic literature on wallet clustering, from early observational studies to wallet fingerprinting." +status: stable +--- + +# A Brief History of Wallet Clustering + +> Originally published as [The Scroll #3: A Brief History of Wallet Clustering](https://spiralxyz.substack.com/p/the-scroll-3-a-brief-history-of-wallet) by Yuval Kogman +> ([@not_nothingmuch](https://x.com/not_nothingmuch)) on Spiral, +> 2025-04-16. + +Our [previous post](wallet-clustering-basics.md) in this series introduced the basic idea behind wallet or address clustering, the trivial case of address reuse, and the merging of clusters based on the common input ownership heuristic (CIOH), also known as the multi-input heuristic. + +Today, we’ll expand on more sophisticated clustering methods, briefly summarizing several notable papers. The content here mostly overlaps with a [live stream](https://www.youtube.com/watch?v=ndgSQlLIfI4) on this topic, which is a companion to this series. Note that the list of works cited is by no means exhaustive. + +## Early Observational Studies - 2011-2013 + +As far as I’m aware, the earliest published academic study that deals with clustering is Fergal Reid and Martin Harrigan’s [An Analysis of Anonymity in the Bitcoin System](https://link.springer.com/chapter/10.1007/978-1-4614-4139-7_10) ([PDF](https://arxiv.org/pdf/1107.4524)). This work, which studies the anonymity properties of bitcoin more broadly, in its discussion of the on-chain transaction graph, introduced the notion of a “User Network” to model the relatedness of a single user’s coins based on CIOH. Using this model, the authors critically examined WikiLeak’s claim that it “[accepts anonymous Bitcoin donations](https://twitter.com/wikileaks/status/80774521350668288).” + +Another study that was not published as a paper was [Bitcoin - An Analysis](https://fahrplan.events.ccc.de/congress/2011/Fahrplan/events/4746.en.html) ([YouTube](https://youtube.com/watch?v=hlWyTqL1hFA)) by Kay Hamacher and Stefan Katzenbeisser, presented at 28c3. They studied money flows using transaction graph data and made some remarkably prescient observations about bitcoin. + +In [Quantitative Analysis of the Full Bitcoin Transaction Graph](https://link.springer.com/chapter/10.1007/978-3-642-39884-1_2) ([PDF](https://eprint.iacr.org/2012/584.pdf)), Dorit Ron and Adi Shamir analyzed a snapshot of the entire transaction graph. Among other things, they note a curious pattern, which may be an early attempt at subverting CIOH: + +> We discovered that almost all these large transactions were the descendants of a single large transaction involving 90,000 bitcoins \[presumably [b9a0961c07ea9a28…](https://mempool.space/tx/b9a0961c07ea9a28bdfa950556bd96f71d9ecf6e42d19c08cb2ee331a30d30cb)\] which took place on November 8th, 2010, and that the subgraph of these transactions contains many strange looking chains and fork-merge structures, in which a large balance is either transferred within a few hours through hundreds of temporary intermediate accounts, or split into many small amounts which are sent to different accounts only in order to be recombined shortly afterward into essentially the same amount in a new account. + +Another early confounding of this pattern was due to MtGox, which [allowed users to upload their private keys](https://en.bitcoin.it/wiki/MtGox/API/HTTP/v1#redeem_a_wallet.dat). Many users’ keys were used as inputs to batch sweeping transactions constructed by MtGox to service this unusual pattern of deposits. The naive application of CIOH to those transactions resulted in cluster collapse, specifically the cluster previously known as [MtGoxAndOthers](https://www.walletexplorer.com/wallet/MtGoxAndOthers) on walletexplorer.com (now known as CoinJoinMess). Ron and Shamir seem to note this, too: + +> However, there is a huge variance in \[these\] statistics, and in fact one entity is associated with 156,722 different addresses. By analyzing some of these addresses and following their transactions, it is easy to determine that this entity is Mt.Gox + +Although change identification is mentioned (Ron & Shamir refer to these as “internal” transfers), the first attempt at formalization appears to be in [Evaluating User Privacy in Bitcoin](https://link.springer.com/chapter/10.1007/978-3-642-39884-1_4) ([PDF](https://eprint.iacr.org/2012/596.pdf)) by Elli Androulaki, Ghassan O. Karame, Marc Roeschlin, Tobias Scherer, and Srdjan Capkun. They used the term “Shadow Addresses,” which these days are more commonly referred to as “change outputs.” This refers to self-spend outputs, typically one per transaction, controlled by the same entity as the inputs of the containing transaction. The paper introduces a heuristic for identifying such outputs to cluster them with the inputs. Subsequent work has iterated on this idea extensively, with several proposed variations. One example based on the amounts in 2 output transactions is if an output’s value is close to a round number when denominated in USD (based on historical exchange rates), that output is likely to be a payment, indicating the other production is the change. + +This early phase of Bitcoin privacy research saw the theory of wallet clustering become established as a foundational tool for the study of Bitcoin privacy. While this wasn’t entirely theoretical, evidential support was limited, necessitating relatively strong assumptions to interpret the observable data. + +## Empirical Results - 2013-2017 + +Although researchers attempted to validate the conclusions of these papers, for example, by interviewing Bitcoin users and asking them to confirm the accuracy of the clustering of their wallets or using simulations as in Androulaki et al.’s work, little information was available about the countermeasures users were utilizing. + +[A fistful of bitcoins: characterizing payments among men with no names](https://dl.acm.org/doi/abs/10.1145/2504730.2504747) (PDFs: [1](https://dl.acm.org/doi/pdf/10.1145/2896384), [2](https://web.archive.org/web/20180417225209id_/http://conferences.sigcomm.org/imc/2013/papers/imc182-meiklejohnA.pdf)) by Sarah Meiklejohn, Marjori Pomarole, Grant Jordan, Kirill Levchenko, Damon McCoy, Geoffrey M. Voelker, and Stefan Savage examined the use of Bitcoin mixers, and put the heuristics to the test by actually using such services with real Bitcoin. On the more theoretical side, they defined a more general and accurate change identification heuristic than previous work. + +In his thesis, [Data-Driven De-Anonymization in Bitcoin](https://www.research-collection.ethz.ch/bitstream/handle/20.500.11850/155286/eth-48205-01.pdf), Jonas Nick was able to validate the CIOH and change identification heuristics using information obtained from a privacy bug in the implementation of BIP 37 bloom filters, mainly used by light clients built with bitcoinj. The underlying privacy leak was described in [On the privacy provisions of Bloom filters in lightweight bitcoin clients](https://dl.acm.org/doi/abs/10.1145/2664243.2664267) ([PDF](https://discovery.ucl.ac.uk/id/eprint/10182350/1/2014-763.pdf)) by Arthur Gervais, Srdjan Capkun, Ghassan O. Karame, and Damian Gruber. The leak demonstrated that the clustering heuristics were rather powerful, a finding which was elaborated on in Martin Harrigan and Christoph Fretter’s [The Unreasonable Effectiveness of Address Clustering](https://ieeexplore.ieee.org/abstract/document/7816867) ([PDF](https://arxiv.org/pdf/1605.06369)). + +Attackers have also been observed sending bitcoin, not through a mixer as in the fistful of bitcoins papers, but small amounts sent to addresses that have already appeared on-chain. This behavior is called dusting or dust[^1] attacks and can deanonymize the victim in two ways. First, the receiving wallet may spend the funds, resulting in address reuse. Second, older versions of Bitcoin Core [used to](https://github.com/bitcoin/bitcoin/pull/18038) rebroadcast received transactions, so an attacker who was also connected to many nodes on the p2p network could observe if any node was rebroadcasting its dusting transactions and that node’s IP address to the cluster.[^2] + +Although [Is Bitcoin gathering dust? An analysis of low-amount Bitcoin transactions](https://link.springer.com/article/10.1007/s41109-023-00557-4) ([PDF](https://link.springer.com/content/pdf/10.1007/s41109-023-00557-4.pdf)) by Matteo Loporchio, Anna Bernasconi, Damiano Di Francesco Maesa, and Laura Ricci offered insights in 2023, exploring dust attacks, the data set they analyzed only extends to 2017. This work looked at the effectiveness of such attacks in revealing clusters: + +> This means that the dust attack transactions, despite being only 4.86% of all dust creating transactions, allow to cluster 66.43% of all dust induced clustered addresses. Considering the whole data set, the transactions suspected of being part of dust attacks are only 0.008% of all transactions but allow to cluster 0.14% of all addresses that would have otherwise remained isolated. + +This period of research was marked by a more critical examination of the theory of wallet clustering. It became increasingly clear that, in some cases, users’ behaviors can be easily and reliably observed and that privacy assurances are far from perfect, not just in theory but also based on a growing body of scientific evidence. + +## Wallet Fingerprinting - 2021-2024 + +Wallet fingerprints are identifiable patterns in transaction data that may indicate using particular wallet software. In recent years, researchers have applied wallet fingerprinting techniques to wallet clustering. A single wallet cluster is typically created using the same software throughout, so any observable fingerprints should be fairly consistent within the cluster.[^3] + +As a simple example of wallet fingerprinting, every transaction has an nLockTime field, which can be used to post-date transactions.[^4] This can be done by specifying a height or a time. When no post-dating is required, any value representing a point in time that is already in the past can be used, typically 0, but such transactions haven’t been post-dated when they were signed. To avoid revealing intended behavior and address the [fee sniping](https://bitcoinops.org/en/topics/fee-sniping/) concern, some wallets will randomly specify a more recent nLockTime value. However, since some wallets always specify a value of 0, when it’s not clear which output of a transaction is a payment and which is change, that information might be revealed by subsequent transactions. For example, suppose all of the transactions associated with the input coins specify nLockTime of 0, but the spending transaction of one of the outputs does not, in this case it would be reasonable to conclude that output was a payment to a different user. + +There are many other known fingerprints. [Wallet Fingerprints: Detection & Analysis](https://ishaana.com/blog/wallet_fingerprinting/) by Ishaana Misra is a comprehensive account. + +Malte Möser and Arvind Narayanan’s [Resurrecting Address Clustering in Bitcoin](https://link.springer.com/chapter/10.1007/978-3-031-18283-9_19) ([PDF](https://arxiv.org/pdf/2107.05749)) applied fingerprinting to the clustering problem. They used it as the basis for refinements to change identification. They relied on fingerprints to train and evaluate improved change identification using machine learning techniques (random forests). + +Shortly thereafter, in [How to Peel a Million: Validating and Expanding Bitcoin Clusters](https://www.usenix.org/conference/usenixsecurity22/presentation/kappos) ([PDF](https://www.usenix.org/system/files/sec22-kappos.pdf)), George Kappos, Haaroon Yousaf, Rainer Stütz, Sofia Rollet, Bernhard Haslhofer and Sarah Meiklejohn extended and validated this approach using cluster data for a sample of transactions provided by a chain analytics company, indicating that the wallet fingerprinting approach is dramatically more accurate than only using CIOH and simpler change identification heuristics. Taking fingerprints into account when clustering makes deanonymization much easier. Likewise, taking fingerprints into account in wallet software can improve privacy. + +A recent paper, [Exploring Unconfirmed Transactions for Effective Bitcoin Address Clustering](https://dl.acm.org/doi/abs/10.1145/3589334.3645684) ([PDF](https://arxiv.org/pdf/2303.01012)) by Kai Wang, Yakun Cheng, Michael Wen Tong, Zhenghao Niu, Jun Pang, and Weili Han analyzed patterns in the broadcast of transactions before they are confirmed. For example, different fee-bumping behaviors can be observed, both via [replacement](https://bitcoinops.org/en/topics/replace-by-fee/) or with [child-pays-for-parent](https://bitcoinops.org/en/topics/cpfp/). Such patterns, while not strictly fingerprints derived from the transaction data, can still be thought of as wallet fingerprints but about more ephemeral patterns related to certain wallet software, observable when connected to the Bitcoin P2P network but not apparent in the confirmed transaction history that is recorded in the blockchain. + +Similar to the Bitcoin P2P layer, the Lightning network’s gossip layer shares information about publicly announced channels. This is not typically framed as a wallet fingerprint but might be loosely considered as such, in addition to the on-chain fingerprint lightning transactions have. Lightning channels are UTXOs, and they form the edges of a graph connecting Lightning nodes, which are identified by their public key. Since a node may be associated with several channels, and channels are coins, this is somewhat analogous to address reuse.[^5] Christian Decker has publicly archived historical graph data. One study that looks at clustering in this context is [Cross-Layer Deanonymization Methods in the Lightning Protocol](https://link.springer.com/chapter/10.1007/978-3-662-64322-8_9) ([PDF](https://arxiv.org/pdf/2007.00764)) by Matteo Romiti, Friedhelm Victor, Pedro Moreno-Sanchez, Peter Sebastian Nordholt, Bernhard Haslhofer, and Matteo Maffei. + +Clustering techniques have improved dramatically over the last decade and a half. Unfortunately, widespread adoption of Bitcoin privacy technologies is still far from being a reality. Even if it was, the software has not yet caught up to the state of the art in attack research. + +## Not The Whole Story + +As we have seen, starting from the humble beginnings of address reuse and the CIOH described by Satoshi, wallet clustering is a foundational idea in Bitcoin privacy that has seen many developments over the years. A wealth of academic literature has called into question some of the overly optimistic characterizations of Bitcoin privacy, starting with WikiLeaks describing donations as anonymous in 2011. There are also many opportunities for further study and for the development of privacy protections. + +Something to bear in mind is that clustering techniques will only continue to improve over time. “\[R\]emember: attacks always get better, they never get worse.”[^6] Given the nature of the blockchain, patterns in the transaction graph will be preserved for anyone to examine more or less forever. Light wallets that use the Electrum protocol will leak address clusters to their Electrum servers. Ones that submit [xpubs](https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki) to a service will leak clustering information of all past and future transactions in a single query. Given the nature of the blockchain analysis industry, proprietary techniques are at a significant advantage, likely benefiting from access to KYC information labeling a large subset of transactions. This and other kinds of blockchain-extrinsic clustering information are especially challenging to account for since, despite being shared with 3rd parties, this information is not made public, unlike clustering based on on-chain data. Hence, these leaks aren’t as widely observable. + +Also, bear in mind that control over one’s privacy isn’t entirely in the hands of the individual. When one user’s privacy is lost, that degrades the privacy of all other users. Through the process of elimination, which suggests a linear progression of privacy decay, every successfully deanonymized user can be discounted as a possible candidate when attempting to deanonymize the transactions of the remaining users. In other words, even if you take precautions to protect your privacy, there will be no crowd to blend into if others don't take precautions, too. + +However, as we shall see, assuming linear decay of privacy is often too optimistic; exponential decay is a safer assumption. This is because divide-and-conquer tactics also apply to wallet clustering, much like in the game of 20 questions. CoinJoins transactions are designed to confound the CIOH, and the topic of the next post will be a paper that combines wallet clustering with intersection attacks, a concept borrowed from the mixnet privacy literature, to deanonymize CoinJoins. + +[^1]: Not to be confused with a different kind of dust attack, [such as this example](https://medium.com/@laurentmt/good-whale-hunting-d3cc3861bd6b) analyzed taking clustering into account by LaurentMT and Antoine Le Calvez. + +[^2]: A notable and somewhat related [attack on Zcash and Monero nodes](https://crypto.stanford.edu/timings/) ([Remote Side-Channel Attacks on Anonymous Transactions](https://www.usenix.org/conference/usenixsecurity20/presentation/tramer) by Florian Tramer, Dan Boneh and Kenny Paterson) was able to link node IP addresses to viewing keys by exploiting timing side channels on the P2P layer. + +[^3]: More precisely: fingerprint distributions should be consistent within a cluster, as some wallets deliberately randomize certain attributes of transactions. + +[^4]: Note for nLockTime to be enforced the nSequence value of at least one input of the transaction must also be non-final, which complicates things both for post-dating and in terms of the different observable patterns this gives rise to. + +[^5]: Channel funds are shared by both parties to the channel but the closing transaction resembles a payment from the funder of a channel. Dual-funded channels may confound CIOH, similarly to [PayJoin](https://payjoin.org/) transactions. + +[^6]: [New Attack on AES - Schneier on Security](https://www.schneier.com/blog/archives/2009/07/new_attack_on_a.html) diff --git a/docs/background/img/faces-of-coinjoin-1-dark.png b/docs/background/img/faces-of-coinjoin-1-dark.png new file mode 100644 index 0000000..fdcd3ee Binary files /dev/null and b/docs/background/img/faces-of-coinjoin-1-dark.png differ diff --git a/docs/background/img/faces-of-coinjoin-1-light.png b/docs/background/img/faces-of-coinjoin-1-light.png new file mode 100644 index 0000000..82b263a Binary files /dev/null and b/docs/background/img/faces-of-coinjoin-1-light.png differ diff --git a/docs/background/img/headaches-of-coinjoin-1-dark.png b/docs/background/img/headaches-of-coinjoin-1-dark.png new file mode 100644 index 0000000..e4f7b23 Binary files /dev/null and b/docs/background/img/headaches-of-coinjoin-1-dark.png differ diff --git a/docs/background/img/headaches-of-coinjoin-1-light.png b/docs/background/img/headaches-of-coinjoin-1-light.png new file mode 100644 index 0000000..1176f76 Binary files /dev/null and b/docs/background/img/headaches-of-coinjoin-1-light.png differ diff --git a/docs/background/img/headaches-of-coinjoin-2-dark.png b/docs/background/img/headaches-of-coinjoin-2-dark.png new file mode 100644 index 0000000..97f8932 Binary files /dev/null and b/docs/background/img/headaches-of-coinjoin-2-dark.png differ diff --git a/docs/background/img/headaches-of-coinjoin-2-light.png b/docs/background/img/headaches-of-coinjoin-2-light.png new file mode 100644 index 0000000..62660a7 Binary files /dev/null and b/docs/background/img/headaches-of-coinjoin-2-light.png differ diff --git a/docs/background/img/headaches-of-coinjoin-3-dark.png b/docs/background/img/headaches-of-coinjoin-3-dark.png new file mode 100644 index 0000000..8b33ee5 Binary files /dev/null and b/docs/background/img/headaches-of-coinjoin-3-dark.png differ diff --git a/docs/background/img/headaches-of-coinjoin-3-light.png b/docs/background/img/headaches-of-coinjoin-3-light.png new file mode 100644 index 0000000..e3db00e Binary files /dev/null and b/docs/background/img/headaches-of-coinjoin-3-light.png differ diff --git a/docs/background/img/headaches-of-coinjoin-4-dark.png b/docs/background/img/headaches-of-coinjoin-4-dark.png new file mode 100644 index 0000000..6e9e9aa Binary files /dev/null and b/docs/background/img/headaches-of-coinjoin-4-dark.png differ diff --git a/docs/background/img/headaches-of-coinjoin-4-light.png b/docs/background/img/headaches-of-coinjoin-4-light.png new file mode 100644 index 0000000..036feae Binary files /dev/null and b/docs/background/img/headaches-of-coinjoin-4-light.png differ diff --git a/docs/background/img/headaches-of-coinjoin-5-dark.png b/docs/background/img/headaches-of-coinjoin-5-dark.png new file mode 100644 index 0000000..07febcd Binary files /dev/null and b/docs/background/img/headaches-of-coinjoin-5-dark.png differ diff --git a/docs/background/img/headaches-of-coinjoin-5-light.png b/docs/background/img/headaches-of-coinjoin-5-light.png new file mode 100644 index 0000000..08b2a99 Binary files /dev/null and b/docs/background/img/headaches-of-coinjoin-5-light.png differ diff --git a/docs/background/img/headaches-of-coinjoin-6-dark.png b/docs/background/img/headaches-of-coinjoin-6-dark.png new file mode 100644 index 0000000..d42cb97 Binary files /dev/null and b/docs/background/img/headaches-of-coinjoin-6-dark.png differ diff --git a/docs/background/img/headaches-of-coinjoin-6-light.png b/docs/background/img/headaches-of-coinjoin-6-light.png new file mode 100644 index 0000000..65a398a Binary files /dev/null and b/docs/background/img/headaches-of-coinjoin-6-light.png differ diff --git a/docs/background/img/intersection-attacks-1.jpg b/docs/background/img/intersection-attacks-1.jpg new file mode 100644 index 0000000..93e7e00 Binary files /dev/null and b/docs/background/img/intersection-attacks-1.jpg differ diff --git a/docs/background/img/intersection-attacks-2-dark.png b/docs/background/img/intersection-attacks-2-dark.png new file mode 100644 index 0000000..9029a60 Binary files /dev/null and b/docs/background/img/intersection-attacks-2-dark.png differ diff --git a/docs/background/img/intersection-attacks-2-light.png b/docs/background/img/intersection-attacks-2-light.png new file mode 100644 index 0000000..e7f09b0 Binary files /dev/null and b/docs/background/img/intersection-attacks-2-light.png differ diff --git a/docs/background/img/intersection-attacks-3-dark.png b/docs/background/img/intersection-attacks-3-dark.png new file mode 100644 index 0000000..7ed8fc4 Binary files /dev/null and b/docs/background/img/intersection-attacks-3-dark.png differ diff --git a/docs/background/img/intersection-attacks-3-light.png b/docs/background/img/intersection-attacks-3-light.png new file mode 100644 index 0000000..37014f4 Binary files /dev/null and b/docs/background/img/intersection-attacks-3-light.png differ diff --git a/docs/background/img/intersection-attacks-4-dark.png b/docs/background/img/intersection-attacks-4-dark.png new file mode 100644 index 0000000..299949c Binary files /dev/null and b/docs/background/img/intersection-attacks-4-dark.png differ diff --git a/docs/background/img/intersection-attacks-4-light.png b/docs/background/img/intersection-attacks-4-light.png new file mode 100644 index 0000000..49dd433 Binary files /dev/null and b/docs/background/img/intersection-attacks-4-light.png differ diff --git a/docs/background/img/intersection-attacks-5-dark.png b/docs/background/img/intersection-attacks-5-dark.png new file mode 100644 index 0000000..748dff4 Binary files /dev/null and b/docs/background/img/intersection-attacks-5-dark.png differ diff --git a/docs/background/img/intersection-attacks-5-light.png b/docs/background/img/intersection-attacks-5-light.png new file mode 100644 index 0000000..a3a4f34 Binary files /dev/null and b/docs/background/img/intersection-attacks-5-light.png differ diff --git a/docs/background/img/intersection-attacks-6-dark.png b/docs/background/img/intersection-attacks-6-dark.png new file mode 100644 index 0000000..b715bea Binary files /dev/null and b/docs/background/img/intersection-attacks-6-dark.png differ diff --git a/docs/background/img/intersection-attacks-6-light.png b/docs/background/img/intersection-attacks-6-light.png new file mode 100644 index 0000000..44c7078 Binary files /dev/null and b/docs/background/img/intersection-attacks-6-light.png differ diff --git a/docs/background/img/intersection-attacks-7-dark.png b/docs/background/img/intersection-attacks-7-dark.png new file mode 100644 index 0000000..11e1654 Binary files /dev/null and b/docs/background/img/intersection-attacks-7-dark.png differ diff --git a/docs/background/img/intersection-attacks-7-light.png b/docs/background/img/intersection-attacks-7-light.png new file mode 100644 index 0000000..50581a0 Binary files /dev/null and b/docs/background/img/intersection-attacks-7-light.png differ diff --git a/docs/background/img/intersection-attacks-8-dark.png b/docs/background/img/intersection-attacks-8-dark.png new file mode 100644 index 0000000..01a9be3 Binary files /dev/null and b/docs/background/img/intersection-attacks-8-dark.png differ diff --git a/docs/background/img/intersection-attacks-8-light.png b/docs/background/img/intersection-attacks-8-light.png new file mode 100644 index 0000000..0eea3a6 Binary files /dev/null and b/docs/background/img/intersection-attacks-8-light.png differ diff --git a/docs/background/img/intersection-attacks-9-dark.png b/docs/background/img/intersection-attacks-9-dark.png new file mode 100644 index 0000000..754c026 Binary files /dev/null and b/docs/background/img/intersection-attacks-9-dark.png differ diff --git a/docs/background/img/intersection-attacks-9-light.png b/docs/background/img/intersection-attacks-9-light.png new file mode 100644 index 0000000..c1a6c3b Binary files /dev/null and b/docs/background/img/intersection-attacks-9-light.png differ diff --git a/docs/background/intersection-attacks.md b/docs/background/intersection-attacks.md new file mode 100644 index 0000000..b3c2ffd --- /dev/null +++ b/docs/background/intersection-attacks.md @@ -0,0 +1,222 @@ +--- +type: Essay +title: "Intersection attacks on CoinJoin anonymity" +description: "Measuring anonymity with entropy, and how intersections of clusters over time erode CoinJoin anonymity sets." +status: stable +--- + +# Intersection Attacks on CoinJoin Anonymity + +> Originally published as [The Scroll #4: Intersection Attacks on CoinJoin Anonymity](https://spiralxyz.substack.com/p/the-scroll-4-intersection-attacks) by Yuval Kogman +> ([@not_nothingmuch](https://x.com/not_nothingmuch)) on Spiral, +> 2025-06-11, licensed under [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/). + +Like the [previous post in The Scroll series](history-of-wallet-clustering.md), this one accompanies and elaborates on a [video presentation](https://www.youtube.com/watch?v=oQk87s0N3BU). + +## Thinking Adversarially about Twenty Questions + +When studying privacy, it’s useful to think of deanonymization as a game. We imagine an adversary with some access to information, and it tries to guess correctly who among a set of candidates was responsible for some event in the system. To defend against the adversary winning, we need to keep it guessing, which could either mean limiting its access to information or using randomness to increase the amount of information it needs to succeed. + +![Source: Guess Who? 1979 edition….Unboxing & Overview](img/intersection-attacks-1.jpg) + +*Source: **[Guess Who? 1979 edition….Unboxing & Overview](https://www.youtube.com/watch?v=5ifCeqYdTG4)*** + +Many readers will be familiar with the game of “[Guess Who?](https://en.wikipedia.org/wiki/Guess_Who%3F)”. This game could be described as a turn-based composition of two instances of the more general game of “[twenty questions](https://en.wikipedia.org/wiki/Twenty_questions)”. In “twenty questions”, you secretly choose an element from a given set, and your opponent tries to guess it correctly by asking you up to 20 yes or no questions. In “Guess Who?”, both sides take turns playing against each other, and the first to guess correctly wins. The set of elements is fixed in “Guess Who?”, consisting of 24 cartoon characters with various distinguishing features, such as their hair color or style. Each character has a unique name that unambiguously identifies them. + +The answers to a yes or no question can be represented as a bit: a 1 or a 0. Twenty bits can express, in base two, any whole number in the range 0 to 1,048,575, which is 2²⁰-1. If a set can be totally ordered, each element in the set may be indexed by its numbered position in the order, which uniquely identifies it. So, twenty bits can uniquely address one of just over a million elements. + +Although 2²⁰ is the maximum number of elements of a set that could be uniquely identified using just the answers to 20 yes or no questions, in real-world situations, 20 answers will often contain less information than that. For most sets and combinations of questions, things will almost certainly not line up perfectly, and not every question will bisect the candidate elements independently of the other questions. The answers to some questions might be biased. Some questions’ answers might correlate with those of other questions. + +Suppose that instead of asking something like “does your character have glasses?” you always ask “alphabetically does your character’s name appear before \[median remaining character’s name\]?”. This is a [binary search](https://en.wikipedia.org/wiki/Binary_search), which will maximize how informative the answer to each question will be: at every step, the median name partitions the set of remaining characters, and the question eliminates one of the two halves. Repeatedly halving the remaining candidates will narrow down the search as quickly as yes or no answers make possible, only a logarithmic number of steps is required, which is much faster than say a linear scan (i.e. checking one by one: “Is it Alice? No? How about Bob? …”). + +
+Source: Binary search - Wikipedia +Source: Binary search - Wikipedia +
+ +*Source: [Binary search - Wikipedia](https://en.wikipedia.org/wiki/Binary_search#/media/File:Binary_Search_Depiction.svg)* + +Bear in mind that if you are playing to win, the point of the game is not to get the most information out of your opponent but to be the first to guess correctly, and it turns out that maximizing the information per answer is actually [not the optimal strategy](https://arxiv.org/pdf/1509.03327) at least when the game is played honestly. Similarly, when using games to study privacy, one must assume the adversary is rational according to its preferences; it’s fairly easy to accidentally optimize for a subtly incorrect outcome. The adversary is playing to win. + +Finally, suppose the players are no longer assumed to be honest. It should be apparent that one can cheat undetectably[^1], instead of choosing an element of the set at the start and then answering honestly, in response to every question, you can always give the answer that would leave the largest number of remaining candidates. Adaptively chosen answers can therefore minimize the rate at which one's opponent obtains useful information to win the game. In this so-called Byzantine setting, the optimal strategy is no longer the same as when players are honest. Here, an opponent’s best response would be to stick with binary search, which limits the advantage of playing adaptively. + +Adaptive “Guess who?” is pretty boring, similar to how tic-tac-toe should always end in a draw if you’re paying attention. To be precise, as we will see in the next section, there are 4.58 bits of information to extract from your maximally adversarial opponent, and the rules of the game can be used to force the opponent to commit to those bits. This means the first player can always win after five questions. The transcript of answers in such games should always consist of uniformly random bits[^2]; anything else would give an edge to one’s opponent. Unfortunately, privacy protections using such adaptivity or added randomness are difficult to build and understand, so actual privacy software is usually significantly harder to analyze than these toy examples. + +## Measuring Anonymity + +### Shannon Entropy + +The [information content](https://en.wikipedia.org/wiki/Information_content) of an answer in “Guess Who?”, also known as its Shannon entropy, quantifies how surprising it is to learn. For example, if you already found out that your opponent’s character is bald, it won’t surprise you to learn that they do not have black hair, and this answer contains no additional information. This wasn’t surprising because, before being told, you could infer that the probability of having black hair was zero. + +Suppose that two options remain from the set of candidates; it’s basically a coin toss, and either of the two options should be equally likely and, therefore, equally surprising. Learning that it’s option A tells you it isn’t B, and equivalently, learning that it’s not B tells you that it must be A, so only one yes or no question is needed, 1 bit of information, to remove all uncertainty. + +This value can be calculated from the probability distribution, which in this binary example is [Bernoulli](https://en.wikipedia.org/wiki/Bernoulli_distribution) with p=1/2. + +First, compute the negation of the base 2 logarithm of the probability of each case, or equivalently invert the probability first and skip the negation: + +$$ +-\log_2(p) = \log_2\left(\frac{1}{p}\right) +$$ + +In both cases + +$$ +log_2(2)=1. +$$ + +These values are then scaled by multiplying these values by their corresponding probabilities, as a sort of weighted average, resulting in a contribution of ½ bits for either case. The sum of these terms, 1 in this case, is the Shannon entropy of the distribution. + +This also works with more than two outcomes. If you start the game by asking “is it \[a random character’s name\]?” you will most likely only learn + +$$ +\frac{1}{24} \log_2(24) \approx 0.191 +$$ + +bits of information if the answer was “no”. + +At that point log₂(23) ≈ 4.52 bits quantify your remaining uncertainty over the 23 equally likely remaining possibilities. On the other hand, if you were lucky and guessed correctly, you will learn the full log₂(24) ≈ 4.58 bits of information, because no uncertainty will remain. + +Just under 5 bits are needed to narrow down to one of 24 characters. 10 bits can identify one in 1024. 20 bits, around one in a million as we saw before. + +Shannon entropy is general enough to quantify non-uniform distributions, too. Not all names are equally popular, so an interesting question is [how much entropy is in a name](https://altmode.org/2015/01/14/how-much-entropy-is-in-a-name/)? The linked post estimates this at roughly 15 bits for US surnames. According to [another paper](https://escholarship.org/content/qt46v58824/qt46v58824.pdf), first names in the US contain approximately 10-11 bits. These estimates imply an upper bound of 26 bits per full name, but remember that a common name like John Smith will contain less information than an uncommon one. Uniquely addressing the entire US population requires 29 bits. + +As of writing, the world population is slowly but surely approaching 2³³ people. 33 is not a very large number. How many bits are in a birthdate? Just an age? Someone’s [city of residence](https://gwern.net/death-note-anonymity)? An IP address? A [favorite movie](https://arxiv.org/pdf/cs.CR/0610105)? A browser’s [canvas implementation](https://coveryourtracks.eff.org/)? A ZIP code? The words in their vocabulary, or the idiosyncrasies of their punctuation? These are tricky questions. Unlike these games and modern cryptography, where secrets are random and preferentially ephemeral, we can’t randomize, expire, or rotate our real-life identifying attributes. + +Furthermore, this personally identifying information often leaks both by necessity and sometimes unnecessarily and unintentionally throughout our lives. We often have to trust people with whom we interact not to reveal this information, whether sharing it to 3rd parties or accidentally leaking it. Perhaps it’s not unlike how we must trust others with our lives, like doctors or professional drivers and pilots. Still, certainly it is not comparable in terms of how necessary it is to trust as a matter of course when it comes to our personal data. + +### Entropist Perspective on Anonymity + +Privacy enhanced systems allow users to hide in a crowd. For example, if you observe a connection to your server from a Tor exit node, for all you know, it’s one of potentially thousands of Tor users that established that connection. Informally, given some event that a deanonymization adversary has observed, perhaps by intercepting a message being transmitted between two nodes in a network, a particular user’s anonymity set refers to the set of potential users to whom that event might be attributed. If the receiver of an anonymous message is taken to be the adversary, then their best guess from a set of candidate senders is the sender’s anonymity set. If this hypothetical system is fully anonymous, then any user is equally likely to have sent the message, apart from the receiver. + +Two influential papers that proposed to measure anonymity in terms of the entropy of the anonymity set were published concurrently: [Towards Measuring Anonymity](https://lirias.kuleuven.be/retrieve/333407) by Claudia Díaz, Stefaan Seys, Joris Claessens, and Bart Preneel and [Towards an Information Theoretic Metric for Anonymity](https://bib.mixnetworks.org/pdf/serjantov2002towards.pdf) by Andrei Serjantov and George Danezis. These works generalize from the assumption that the adversary can guess the correct user from an anonymity set no better than chance, to a model that accounts for non-uniform probability distributions over this set. Both propose the quantification of anonymity set sizes in terms of bits of entropy. + +When the anonymity set is perfectly symmetric, only the uniform distribution makes sense, so converting the anonymity set size to bits is just a matter of computing a log₂(n) where n is the size of the set. For example, 1024 equiprobable elements in a set have 10 bits of entropy in their distribution. + +When the distribution is not uniform, the entropy of the distribution decreases. For example, if either heads or tails is possible, but there’s a ¼ probability of heads, ¾ of tails, the total entropy of this distribution is only + +$$ +\frac{1}{4} \log_2(4) \ + \frac{3}{4} \log_2(\frac{4}{3}) = \frac{1}{4} 2 + \frac{3}{4} 0.415...=0.811... +$$ + +bits instead of a full bit. This quantifies the uncertainty represented in a probability distribution; the outcome of flipping this bent coin is comparatively less uncertain than that of a fair coin. + +Shannon entropy is a special case of an entire [family of entropy definitions](https://en.wikipedia.org/wiki/R%C3%A9nyi_entropy). It characterizes the average information content in a message (a yes or no answer, or more generally) drawn from a probability distribution over possible messages. A more conservative estimate might use [min-entropy](https://en.wikipedia.org/wiki/Min-entropy), which considers only the highest probability element instead of calculating the arithmetic mean, quantifying the worst-case scenario. In this post, we’ll stick to Shannon entropy. For a more in-depth discussion and a nuanced interpretation of the entropist perspective, Paul Syverson’s [Why I'm not an Entropist](https://www.freehaven.net/anonbib/cache/entropist.pdf) is a thoughtful read. + +## Intersections + +In [k-anonymity: a model for protecting privacy](https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=2ab47454f59d9d8e55d4d8a69530562a3690794a), Latanya Sweeney reviews some of her prior results as motivation, results which demonstrated re-identification of “anonymized” data. Individually, each attribute in a data set associated with an entry, such as a date of birth, might seem to reveal very little about the subject of that entry. But like the yes or no questions from the game, only a logarithmic amount of information is needed, in other words, combinations of surprisingly small numbers of attributes will often be sufficient for re-identification: + +> For example, a finding in that study was that 87% (216 million of 248 million) of the population in the United States had reported characteristics that likely made them unique based only on {5-digit ZIP, gender, date of birth}. Clearly, data released containing such information about these individuals should not be considered anonymous. + +As a rough estimate, a string of 5 digits would have log₂(10⁵) ≈ 16.6 bits of max entropy, but there are fewer ZIP codes than that, log₂(4.3 x 10⁴) ≈ 15.4, and keep in mind that the population is not uniformly distributed over ZIP codes, so 13.8 would be a [better estimate](https://www.johndcook.com/blog/2018/03/02/bits-of-information-in-a-us-zip-code/). A gender field would usually contain slightly more than 1 bit of information in most circumstances, because even if non-binary genders are represented, the majority of entries will be male or female. That said, entries with non-binary values would reveal a lot more than 1 bit about the subject of that entry. A date of birth is also tricky to estimate without looking at the distribution of ages. + +Ignoring February 29th and assuming uniformly distributed birthdays and 2-digit birth year, the entropy would be log₂(365 x 10²) ≈ 15.1. Again, a more [realistic estimate](https://www.johndcook.com/blog/2018/03/02/bits-of-information-in-age-birthday-and-birthdate/) is available, 14.9 bits. Taken together, the more conservative estimates total roughly 29.7 bits. For comparison, the entropy of a uniform distribution over the US population at the time is log₂(248 x 10⁶) ≈ 27.9 bits, or log₂(342 x 10⁶) ≈ 28.4 with up-to-date [figures](https://www.census.gov/popclock/). + +The following diagram from the paper will probably look familiar to anyone who has spent some time learning what an “inner join” is in SQL. It illustrates a different example where Sweeney linked medical records to the voter registration list using the same fields, identifying then-Massachusetts Governor William Weld’s specific record in an “anonymized” medical dataset: + +
+Source: k-anonymity: a model for protecting privacy +Source: k-anonymity: a model for protecting privacy +
+ +*Source: k-anonymity: a model for protecting privacy* + +
+Source: File:SQL Join - 07 A Inner Join B.svg - Wikimedia Commons +Source: File:SQL Join - 07 A Inner Join B.svg - Wikimedia Commons +
+ +*Source: [File:SQL Join - 07 A Inner Join B.svg - Wikimedia Commons](https://commons.wikimedia.org/wiki/File:SQL_Join_-_07_A_Inner_Join_B.svg)* + +This kind of Venn diagram, with two sets represented by two overlapping circles and the overlapping part highlighted, typically represents an intersection between two sets. Sets are unordered collections of elements, such as rows in a database, numbers, or anything else that can be mathematically defined. The intersection of two sets is the set of elements that are present in both sets. So, for example, within the voter registration list, we might talk about the subset of all entries whose ZIP code is 12345, and the set of all entries whose birth date is January 1, 1970. The intersection of these two subsets is the subset of entries whose ZIP code is 12345 *and* whose date of birth is January 1, 1970. In the Governor's case, there was just one entry in the subset of entries whose attribute values matched his attributes in the voter registration list. + +For data sets with different structures, there’s a small complication: if we think of them as sets of rows, then their intersection would always be empty, because the rows would have different shapes. When computing the inner join of two database tables, only the values of columns which are present in both tables are in some sense intersected by specifying something like `JOIN ON a.zip = b.zip AND a.dob = a.dob`, or the less portable `USING(zip, dob)` syntax, but these intersecting values are related to the rows they came from, so the overall structure of linking two data sets is a bit more involved. Note that Sweeney’s diagram depicts the intersection of the columns of the data sets, emphasizing the more primary problem, which is that attributes included in the “anonymized” data set unintentionally had a non-empty intersection with the attributes of other publicly available data sets. + +On the applied side of the k-anonymity model, the procedures for anonymizing datasets described in the paper have fallen out of favor due to some weaknesses discovered in subsequent work entitled [Attacks on Deidentification's Defenses](https://www.usenix.org/conference/usenixsecurity22/presentation/cohen) by Aloni Cohen. That central idea in k-anonymity is to ensure that for every possible combination of attributes, there are at least k rows containing every specific combination in the data, which suggests log₂(k) additional bits of information would be needed to identify an entry from its congruent ones. The de-identification procedure suggested for ensuring this was the case was to redact or generalize in a data dependent way, for example, drop the day from a date of birth, keeping the year and month, or even only the year if that’s not enough. Cohen’s work shows how easy it is to underestimate the brittleness of privacy, because even discarding information until there’s k of every combination, the redaction process itself leaks information about the statistics of the unredacted data set. Such leaks, even if very subtle, will not only add up over time, but they will typically compound. Accounting for privacy loss using bits, which are a logarithmic scale, perhaps helps provide a better intuition for the typically exponential rate of decay of privacy. + +## Intersection Attacks + +In a mixnet such as [Mixmaster](https://en.wikipedia.org/wiki/Mixmaster_anonymous_remailer) or, more recently, [Katzenpost](https://katzenpost.network/) or [Nym](https://nym.com/)[^3], participants send messages through a series of relays, and each relay shuffles messages before relaying them to the next hop. A global passive adversary can observe all messages being received by all nodes. Onion encryption is used to ensure pre-and post-mix messages are unlinkable, so if at least one relay is honest, the adversary can’t distinguish any individual message from a shuffled batch. + +Over time, however, activity patterns can still be observed, and in fact are unavoidable. [Statistical disclosure attacks on mixnets](https://sci-hub.se/https://link.springer.com/chapter/10.1007/978-3-540-30114-1_21) by George Danezis and Andrei Serjantov were built on prior work that already considered intersections of sets of possible recipients when attempting to link senders to receivers on a mixnet. The attack described in the paper allows an adversary to infer the set of receivers a targeted user is most likely communicating with by intersecting the sets of users that received messages at all during intervals when the targeted user was transmitting messages. Note that there is a combinatorial explosion hidden in the way I formulate this problem here: the output ranges over all possible subsets of the receivers, of which there are 2ⁿ for a set of n receivers. This is an intractable number of combinations even for n of a few dozen users. A major result from this work was an efficient approximation technique, which made these kinds of intersection attacks much more practical. + +## Intersection Attacks on CoinJoin Anonymity + +In their paper [When the cookie meets the blockchain: Privacy risks of web payments via cryptocurrencies](https://arxiv.org/pdf/1708.04748) Steven Goldfeder, Harry Kalodner, Dillon Reisman, and Arvind Narayanan describe two independent but related attacks. Perhaps more importantly, they also make a very compelling case for the brittleness of privacy more broadly, by clearly demonstrating how privacy leaks can compound. + +In Bitcoin, a natural definition of an anonymity set for a coin is the set of wallet clusters (see the [previous](wallet-clustering-basics.md) [posts](history-of-wallet-clustering.md)) into which the coin could plausibly be merged. The anonymity set is non-trivial if there is more than one candidate cluster, in which case merging would be contingent on obtaining additional information. New transactions might introduce uncertainty, necessitating the creation of new clusters for outputs that can’t be merged into any existing cluster (yet). On the other hand, new transactions and out-of-band information can also remove uncertainty and facilitate the merging of clusters. Most commonly, if the multi-input heuristic is considered valid for such a new transaction, then the clusters of the input coins will be merged. However, as we saw before, many heuristics exist, some of which are alarmingly accurate. + +Suppose that Alice received some Bitcoin into a wallet under her control. Some might have been withdrawn from an exchange (presumably with KYC information). Maybe a friend paid her back for lunch. Maybe she sold her car. After making several transactions, Alice realizes that her transaction history is visible to all, and pretty straightforward to interpret, but soon she will need to make not just one, but two separate transactions, with stronger privacy assurances than she has been relying on so far. + +After learning a bit about privacy, Alice decides to use a wallet that supports CoinJoin. Over several CoinJoin transactions she spends her existing coins, obtaining replacement coins that apparently have a non-trivial anonymity set. Before CoinJoining, her wallet was likely clusterable. After CoinJoining, each UTXO she now has can’t be assigned to any specific cluster, since other users’ wallet clusters are also implied in the various CoinJoin transactions. + +The intuition behind CoinJoin privacy is that since multiple inputs belonging to different users are used to create outputs that all look the same, no one output can be linked to a specific input. This is somewhat analogous to a mixnet, where each CoinJoin transaction is a relay and the “messages” being mixed are the coins themselves. This analogy is very simplistic, there are many complications when implementing CoinJoins that cause it to break down, but we will ignore these nuances in this post and give Alice’s chosen CoinJoin wallet the benefit of the doubt and assume that Alice can always successfully spend just one input into each CoinJoin, and that this results in perfect mixing of her funds with those of the other parties to the CoinJoin. Under these assumptions, if there are k equivalent outputs in a CoinJoin transaction, and k separate clusters for the inputs, then each output’s anonymity set should have log₂(k) bits of entropy when this transaction is created. + +### Post CoinJoin Clustering + +The stage is now set for the first attack described in the paper. This attack was made possible by inclusion of 3rd party resources, e.g. a payment processor’s javascript, on merchant websites. Supposing the payment address used for the transaction is revealed to the third party, that would link Alice’s web session to her on-chain transaction. The paper is from 2017, so the specifics of web related leaks are somewhat dated by now, but the principle underlying this concern is as relevant as ever. + +Alice uses one of her CoinJoin UTXOs to make the first of those privacy-demanding transactions. Assuming no semantic leaks (such as a billing address related to a purchase) or metadata leaks (perhaps she [broadcasts using Tor](https://github.com/bitcoin/bitcoin/pull/29415)), this transaction should preserve the privacy Alice obtained from the prior CoinJoin transaction. As drawn here, that would be 1 bit’s worth. The colors of inputs or outputs indicate the cluster they are already assigned to. Alice’s coins are in red, and gradients represent ambiguity: + +
+ + +
+ +While the first transaction does not reveal much on its own, suppose Alice makes another transaction. Let’s say it’s with a different merchant, but one that uses the same payment processor as the first merchant. Naively, it would appear that the following diagram represents the privacy of Alice’s payment transactions, and that the adversary would need 2 bits of additional information, 1 for each transaction, to attribute them both to Alice’s cluster: + +
+ + +
+ +Although Alice intends this to be unlinkable to the first transaction, she might not realize her web browsing activity is being tracked. The paper showed that this kind of tracking was not just possible but even practical, and can reveal to a 3rd party that the two transactions can be clustered even though they don’t appear related on-chain. Visually we can represent this clustering with additional colors: + +
+ + +
+ +Web tracking, as discussed in the paper, is just one of many ways information that facilitates clustering can leak. For example, website breaches can result in purchase records being made public, even years after the fact. In at least [one example](https://twitter.com/hdevalence/status/1578128350958059520), legal proceedings, which are supposed to protect victims, ended up exposing them to even more harm by needlessly revealing information about the on-chain transactions of customers through improper redaction of the transacted amounts. The previous post on the history of wallet clustering provides several additional examples. + +Specifically in the context of CoinJoins, a typical way that this sort of linkage could occur is when the change outputs of post-mix payment transactions are subsequently CoinJoined in a manner that causes them to be linkable by clustering the inputs. This is also known as the toxic change problem[^4], which is illustrated in the next diagram. Note that white doesn’t represent a single cluster, just lack of clustering information in this example. + +
+ + +
+ +If the coordinator of the supposedly “trustless” CoinJoin protocols is malicious, then even [attempting to CoinJoin](https://groups.google.com/g/bitcoindev/c/CbfbEGozG7c) may link the transactions, even if this doesn’t become self-evident on-chain. The consequences are the same as the attack described in the paper, except that a CoinJoin coordinator can also pretend that some participants failed to submit their signatures in time, actively but covertly, or at least deniably disrupting rounds to obtain more clustering information. + +### Intersecting Antecessor Clusters + +Unfortunately for Alice, the story doesn’t end there. What the paper showed next was that given such linking of post-CoinJoin transactions, regardless of how this clustering was learned, an intersection attack on the privacy of the CoinJoin transactions themselves also becomes possible. + +It’s as if the adversary is playing “Guess Who?”, given a payment transaction, it tries to guess where the funds originated from. Consider the set of inputs for each CoinJoin transaction. Every one of the spent coins is assigned to some cluster. Every one of the CoinJoin transactions Alice participated in has an input that is linkable to one of *her* clusters. The privacy of such transactions derives from being linked to a large number of otherwise unrelated clusters. Armed with knowledge that post-CoinJoin transactions link multiple CoinJoin outputs together, the adversary can compute the intersection of the sets of associated clusters. How often will it be the case that a random individual user participated in *every* transaction that Alice did? What about more than one? Not very often. And suppose the intersection contains a unique cluster, which could often eventually be the case. In that case, the adversary will be able to link Alice’s transactions to each other and her pre-CoinJoin transaction history, effectively undoing the mix. + +Visually, this combines the inferences of previous diagrams. For each coin in the purple cluster of the last two diagrams, we can intersect the sets of colors in the gradients depicted in the diagram before that: + +$$ +\{ \textrm{Red}, \textrm{Green} \} \cap \{ \textrm{Red}, \textrm{Blue} \} +$$ + +Only Alice’s red cluster is in the intersection so that the purple cluster can be merged into the red one. Not only do Alice’s clusters merge, since this example only has two user CoinJoin transactions, the remaining clusters also be merged with their ancestors by process of elimination, so Alice’s linkable payments would also potentially deanonymize a hypothetical Bob and Carol in this particular case: + +
+ + +
+ +This suggests that even if CoinJoins functioned like a perfect mix (which they don’t, but that’s a topic for a later post), insufficient post-mix transaction privacy can additionally undermine the privacy of the prior CoinJoin transactions, and much more rapidly than seems intuitive. The graph structure, which connects bitcoin transactions, contains a wealth of information available to a deanonymization adversary. + +Privacy concerns are often downplayed, perhaps due to defeatist attitudes in light of the challenges of preventing or even controlling privacy leaks. Hopefully awareness will improve, and things will play out like they did in cryptography in previous decades, whether it’s no longer shipping weak [“export” crypto](https://en.wikipedia.org/wiki/Export_of_cryptography_from_the_United_States#PC_era), or how [timing side channels](https://en.wikipedia.org/wiki/Timing_attack) were mostly ignored at first, but are now widely understood to be practically exploitable and implementations that don’t take them into account are considered insecure. That said, it’ll always be more challenging, in cryptography we have more opportunities to limit the harm of unintended exposure by preferring ephemeral keys over long-term ones, or at least rotating long-term keys periodically. Sadly, the closest analogue of rotating keys I can think of in privacy is witness protection programs, a rather extreme and costly measure, and far from perfectly effective. + +In future posts, we will explore some of the challenges of CoinJoin privacy in the real world, and how not only the intersection attack by third parties can be addressed, but also how to deal with untrusted counterparties being included in the threat model. + +[^1]: A cryptographic commit-reveal scheme can prevent this kind of adaptive strategy from being used. + +[^2]: Except the question before last, which would be uniformly distributed if the game had 32 characters instead of 24. + +[^3]: Note that Tor is not a mixnet, it deals with long lived connections, not independent messages, nor are the packets associated with individual connections delayed or shuffled like messages in a mixing stage. + +[^4]: This term is overloaded, and doesn’t always refer to this definition. diff --git a/docs/background/wallet-clustering-basics.md b/docs/background/wallet-clustering-basics.md new file mode 100644 index 0000000..6b5905d --- /dev/null +++ b/docs/background/wallet-clustering-basics.md @@ -0,0 +1,36 @@ +--- +type: Essay +title: "Wallet clustering basics" +description: "How chain analysis links coins to a common owner, starting from address reuse and the common input ownership heuristic." +status: stable +--- + +# Wallet Clustering Basics + +> Originally published as [The Scroll #2: Wallet Clustering Basics](https://spiralxyz.substack.com/p/the-scroll-2-wallet-clustering-basics) by Yuval Kogman +> ([@not_nothingmuch](https://x.com/not_nothingmuch)) on Spiral, +> 2025-03-12. + +The Bitcoin transaction graph has various observable patterns. Some of these patterns have been studied and used to link coins from the same wallet, both in theory and practice. + +Every transaction consists of a list of inputs (where the sats are taken from) and outputs (where the input sats are distributed). **Inputs** refer to the outputs of previous transactions, such as connecting transactions. **Outputs** lock some amount of Bitcoin with certain spending conditions (i.e., the "address," public key, or output script). Linking coins means identifying the entity that controls the keys to a collection of transaction outputs, spent or unspent. + +Section 10 of the Bitcoin Whitepaper, "Privacy,” briefly discusses linking: + +> A new key pair should be used for each transaction to keep them from being linked to a common owner. + +When the same public key controls more than one coin, these coins are trivially linked since only one entity is supposed to know the private key. + +However, address reuse is not the only concern. The paper continues: + +> Some linking is still unavoidable with multi-input transactions, which necessarily reveal that their inputs were owned by the same owner. + +This is often referred to as the "common input ownership heuristic," **CIOH**, or the "multi-input heuristic." It's only a heuristic because, unlike the implication in the quote above, counter-examples exist. Although it isn't always true, it often is. + +Over the years, more sophisticated methods for clustering have been developed, for example, telling apart change outputs from payments or using larger structures in the transaction graph than just individual transactions. Some of these have been described in academic work, while others remain proprietary. Improved methods can link to more coins or avoid so-called "cluster collapse," where coins belonging to different users are incorrectly connected. Commercial offerings often benefit from additional sources of information, such as KYC data; they don’t necessarily depend on just the privacy leaks that occur in the Bitcoin protocol, but clustering is still the central theme. + +This motivates an adversarial framing of privacy, where a deanonymization attack attempts to assign coins to clusters. From this perspective, defending privacy means making it more difficult for the adversary to succeed in correctly assigning coins to clusters. The most notable examples involve collaborative transaction construction, whether it is overtly difficult to guess, as in CoinJoin, or covertly as in PayJoin, or perhaps most prominently just a part of how the software works as with Lightning node transactions, in all cases the simplistic assumption breaks down necessitating a more nuanced analysis. + +The adversarial framing also makes it explicit that different adversaries have different capabilities, with the appropriate adversarial model depending on the user’s threat model: Are you more worried about surveillance by an oppressive government or snooping by your transaction’s counterparties? + +Future posts will go into more detail on these subjects. diff --git a/docs/index.md b/docs/index.md index 18aee53..eefbbbb 100644 --- a/docs/index.md +++ b/docs/index.md @@ -2,6 +2,14 @@ okf_version: "0.2" --- +# Background + +* [Wallet clustering basics](background/wallet-clustering-basics.md) - How chain analysis links coins to a common owner, starting from address reuse and the common input ownership heuristic. +* [A brief history of wallet clustering](background/history-of-wallet-clustering.md) - A survey of the academic literature on wallet clustering, from early observational studies to wallet fingerprinting. +* [Intersection attacks on CoinJoin anonymity](background/intersection-attacks.md) - Measuring anonymity with entropy, and how intersections of clusters over time erode CoinJoin anonymity sets. +* [The many faces of CoinJoins](background/faces-of-coinjoin.md) - What "CoinJoin" refers to: equal amount CoinJoins, multiparty batching, PayJoin and Lightning. +* [The many headaches of CoinJoin](background/headaches-of-coinjoin.md) - Why designing a CoinJoin that overtly optimizes for privacy is harder than the folk anonymity set model suggests. + # BIP Drafts * [PSBT Scrubber](bip-drafts/scrubber.md) - Which PSBT fields can be shared with trusted and untrusted parties such that privacy is preserved. diff --git a/theme/diagrams.css b/theme/diagrams.css index bb71486..b07044b 100644 --- a/theme/diagrams.css +++ b/theme/diagrams.css @@ -3,7 +3,8 @@ once per theme and swapped here; graphviz keeps whatever its .dot source asks for, with only graphviz's own defaults lifted for the dark themes. */ .mdbook-graphviz-output, -.mermaid { +.mermaid, +.chart-src { overflow-x: auto; margin: 1rem 0; } @@ -11,7 +12,8 @@ /* Scale wide graphs down to the content column instead of clipping them; the svg viewBox keeps them sharp at any size. */ .mdbook-graphviz-output svg, -.mermaid svg { +.mermaid svg, +.chart-src img { display: block; max-width: 100%; height: auto; @@ -23,14 +25,16 @@ only just overflow still fit without scrolling. */ @media (max-width: 700px) { .mdbook-graphviz-output, - .mermaid { + .mermaid, + .chart-src { margin-left: calc(-1 * var(--page-padding, 15px)); margin-right: calc(-1 * var(--page-padding, 15px)); padding: 0 var(--page-padding, 15px); } .mdbook-graphviz-output svg, - .mermaid svg { + .mermaid svg, + .chart-src img { max-width: none; } } @@ -67,3 +71,37 @@ html:is(.coal, .navy, .ayu) .mdbook-graphviz-output [fill="black"] { html:is(.coal, .navy, .ayu) .mdbook-graphviz-output text:not([fill]) { fill: #b7b7b7; } + +/* Each chart has a light and a dark file, marked for GitHub by its URL + fragment; book themes select one the same way. */ +.chart-src img[src$="#gh-dark-mode-only"] { + display: none; +} + +/* Print the light palette even when the reader selected a dark book theme. */ +@media screen { + html:is(.coal, .navy, .ayu) .chart-src img[src$="#gh-dark-mode-only"] { + display: block; + } + + html:is(.coal, .navy, .ayu) .chart-src img[src$="#gh-light-mode-only"] { + display: none; + } +} + +/* Without JavaScript, mdBook darkens the page by prefers-color-scheme alone. */ +@media screen and (prefers-color-scheme: dark) { + html:not(.js) .chart-src img[src$="#gh-dark-mode-only"] { + display: block; + } + + html:not(.js) .chart-src img[src$="#gh-light-mode-only"] { + display: none; + } +} + +@media print { + .chart-src { + break-inside: avoid; + } +}