(WIP) BIP324 Notes
My primary 2021 Bitcoin Core project is to try and enable authenticated connections to seeds for nodes first connecting to the network. You can read more about the goal of the project here. I explored the following approaches to get to authenticated seeds:
- Adding a cryptographic signature into DNS results (issues: asymmetric keys support needed in bitcoin-seeder, DNS results are paginated, not all results are used by the Bitcoin Core node)
- Format preserving encryption (eg. Feistel network) could be used. This would overcome most of the issues with just adding a signature to DNS results since the authentication is inline, but the problem of key exchange still persists. I was not able to find any reasonable ways to do FPE with asymmetric keys.
- Leveraging BIP324 p2p AEAD(Authenticated Encryption with Associated Data) for authenticating the seeds (this is the leading contender)
This page is a work-in-progress - it contains my notes on BIP324 and open questions.
Why is AEAD useful for the Bitcoin Core p2p network?
The current Bitcoin p2p protocol (v1) is unencrypted, and unauthenticated. This makes it susceptible to BGP(border gateway protocol) and MITM(man-in-the-middle) attacks. The v1 protocol does use a double-SHA256 checksum truncated to 4 bytes. This checksum is (1) more computationally expensive than it needs to be and (2) only protects against random, non-malicious networking errors and not against MITM attacks (an MITM attacker can replace the content of the p2p message as well as the checksum). As we’ll see, #1 is what creates the opportunity for v2 to be faster and more secure than v1.
What is BIP324?
BIP324 AEAD encryption and the v2 protocol is compatible with the v1 protocol and is designed to avoid segmenting the network. A v2 node signals its ability to accept v2 connections using the service flag NODE_P2P_V2 = (1 << 11)
. This flag is available to peers via the seed filtering mechanism and is also made available in the p2p ADDR message. Per BIP324, a v2 node must accept encrypted connections via the handshake mechanism explained below.
Cipher suite
BIP324 combines the Chacha20 symmetric cipher for encryption and the Poly1305 MAC(message authentication code). This cipher suite is also used in TLS 1.3 AEAD. However, because https
uses SSL Certificate Authorities, and Bitcoin can’t, despite using Chacha20-Poly1305, the v2 protocol will remain vulnerable to MITM attacks. The attacks on v2 however become observable as we’ll see below.
The v2 protocol uses a Diffie-Hellman handshake to establish a shared secret between the peers. It then uses HKDF extract-then-expand to deterministically (1) make sure that the bits in the shared secret are from a distribution close to uniform and (2) derive four keys and an encryption session id.
Handshake
The initiator generates an ephemeral secp256k1
(no new dependencies!) key pair (pub_A, priv_a)
where pub_A = priv_a * G
and sends just the 32 bytes of the public key pub_A
to the peer. This key must NOT:
- start with the 4 magic bytes all v1 p2p messages begin with (
0xF9BEB4D9
) so the peer can know a v2 handshake is being initiated. - be even. BIP324 requires that the public keys must be odd (starting with
0x03
instead of0x02
. TODO: I think the BIP has odd-even mixed in this revision - I’ll have to verify what the code actually does). Because this is fixed we only need the 32bytes of the x-coordinate rather than a 33 byte compressed SEC format or a 65 byte uncompressed SEC format. The reasoning from this is in the BIP324 comment thread:
Using a standard 33byte pubkey with 0x02 or 0x03 at the beginning would make a handshake trivially identifiable by someone listening on the wire. If we only use ODD pubkeys, the handshake looks pseudo-random (it is still easy to identify a bitcoin handshake by checking if the first 32 bytes is a valid secp256k1 curve point). It is a low hanging fruit that improves the identifiability-robustness slightly with almost no cost.
There is nothing else sent in the initiation. No headers, no checksum, no payload. The entire TCP message is just the 32 bytes of the public key. The other peer does exactly the same, and responds with its own 32-byte secp256k1
public key pub_B
. Once both peers know the other public key, they can each use their private key with that to generate a shared secret that we will call ECDH_KEY
. So long as an attacker knows neither private key, and the cannot solve the discrete log problem over elliptic curves, the shared secret remains a secret.
ECDH_KEY = pub_A * priv_b = pub_B * priv_a = priv_a * priv_b * G
Key extraction from shared secret
Once the shared secret is established, both peers run a deterministic process to generate 4 keys and 1 session id: Keys K1A
and K2A
are used by peer A to send messages to peer B. Keys K1B
and K2B
are used by peer B. At first I thought the two keys per peer were one for the encryption and one for the MAC. But that is not the case. From BIP324:
The instance keyed by K_1 is a stream cipher that is used for the per-message metadata, specifically for the poly1305 authentication key as well as for the length encryption. The second instance, keyed by K_2, is used to encrypt the entire payload.
Two separate cipher instances are used here so as to keep the packet lengths confidential (best effort; for passive observing) but not create an oracle for the packet payload cipher by decrypting and using the packet length prior to checking the MAC. By using an independently-keyed cipher instance to encrypt the length, an active attacker seeking to exploit the packet input handling as a decryption oracle can learn nothing about the payload contents or its MAC (assuming key derivation, ChaCha20 and Poly1305 are secure). Active observers can still obtain the message length (ex. active ciphertext bit flipping or traffic shemantics analysis)
This two cipher instance solution seems to be derived from [email protected] into ChaCha20-Poly1305@bitcoin. The reasoning for using two instances for openssh can be read here. Essentially there are three ways to do AEAD:
- Encrypt-then-MAC
- MAC-then-encrypt
- Encrypt-and-MAC
The last two are insecure because they require decryption before checking the MAC and allows for an active attacker to build timing or padding oracles. Encrypt-and-MAC is also insecure because a MAC has no confidentiality requirement. However, the only acceptable Encrypt-then-MAC requires that the payload length be in the clear which still allows for padding oracles and traffic analysis. We can avoid that by using one stream cipher instance to encrypt the length of the payload and to generate a MAC, and then a second instance to encrypt the payload.
To understand the choice of two stream cipher instances in more details, read the [email protected] 1.5 spec. There you’ll see:
Two separate cipher instances are used here so as to keep the packet lengths confidential but not create an oracle for the packet payload cipher by decrypting and using the packet length prior to checking the MAC. By using an independently-keyed cipher instance to encrypt the length, an active attacker seeking to exploit the packet input handling as a decryption oracle can learn nothing about the payload contents or its MAC (assuming key derivation, ChaCha20 and Poly1305 are secure).
TODO: I don’t quite understand how encrypting payload length helps if the v2 message structure implies payload length = tcp_packet_size - 16(mac) - 3 bytes encrypted length
The keys are extracted from ECDH_KEY
using HKDF
extract-then-expand steps as shown below.
Extract the pseudo-random key(PRK) from potentially weak input key material(IKM). This is especially important as a node has no way to make sure that the peer is using sufficient entropy to generate its key:
PRK = HKDF_EXTRACT(hash=SHA256, salt="BitcoinSharedSecret||INITIATOR_32BYTES_PUBKEY||RESPONDER_32BYTES_PUBKEY||NETWORK_MAGIC", ikm=ECDH_KEY).
This makes PRK
deterministically derived from ECDH_KEY
but from a closer to uniform distribution. KDF extraction helps make sure exploits don’t occur in case the initial key pairs were derived with insufficient entropy.
Expand into deterministic output key material (OKM):
K1A = HKDF_EXPAND(prk=PRK, hash=SHA256, info="BitcoinK_1_A", L=32)
K1B = HKDF_EXPAND(prk=PRK, hash=SHA256, info="BitcoinK_1_B", L=32)
K2A = HKDF_EXPAND(prk=PRK, hash=SHA256, info="BitcoinK_2_A", L=32)
K2B = HKDF_EXPAND(prk=PRK, hash=SHA256, info="BitcoinK_2_B", L=32)
SID = HKDF_EXPAND(prk=PRK, hash=SHA256, info="BitcoinSessionID", L=32).
Session ID
SID
, the last derived value above is deterministically derived from the shared secret. So if both peers can communicate over a secret channel and compare the session id, they can be assured there is no man-in-the-middle attack being executed. However, in the existing Bitcoin network, these is no such secure channel. The claim is that the possibility of checking the session IDs and observability of MITM attacks dissuades such attacks.
TODO: MITM not an issue with authenticated seeds because one public key would be in the codebase and cannot be then MITMed.
Chacha20-Poly1305@bitcoin
- TODO: How is the cipher suite used in bitcoin optimized for bitcoin p2p messages compared to the IETF spec?
FAQs
Can’t I just use tor or a VPN?
Yes, you could encrypt p2p traffic with an OSI level technique but that requires awareness, expertise and some degree of adversarial thinking that shouldn’t be a requirement to be a full node operator.
Will encryption mean more CPU to run a full node?
- TODO: Talk about double-SHA256 v/s AEAD times. JS slide.
Will the v2 protocol require more bandwidth for a full node?
- TODO: v1/v2 message structure comparison goes here.
Isn’t Diffie-Hellman key exchange susceptible to man-in-the-middle attacks?
- TODO: session id helps if another secure channel is available.
Why is the network magic not present in the p2p v2 message structure?
The network bytes are present in the v1 message structure to ensure that two nodes that intend to be on different networks do not end up talking to each other. Since the derivation of the PRK in the HKDF_EXTRACT step (using the ECDH_KEY as weak input key material) uses the network magiv bytes in the salt, two nodes that intend to be on different networks will end up generating different PRKs and will not be able to talk to each other.
BIP324 timeline and PRs
- March 2016: Jonas Schnelli proposes BIP151
- March 2019: BIP 324 created. Supersedes BIP151.
- December 2019: Jonas Schnelli presents at Breaking Bitcoin. Slides.
Critical path Pull Requests:
Bitcoin Core
- August 2018: (closed; useful for archival)#14032 is opened by jonasschnelli. This PR is a complete implementation that is later broken down into smaller PRs mentioned below. This original proof-of-concept includes the crypto, the refactors as well as the v2 protocol implementation.
- August 2018: (closed; needs revival)#14049 is opened by jonasschnelli. This PR adds a function to generate the shared secret given the node’s private key and the peer’s public key using the libsecp256k1 elliptic curve.
- August 2018: (closed; superseded)#14050 is opened to add Chacha20-Poly1305@openssh. In 3⁄2019, Jonas Schnelli reports this PR is superseded by #15512, #15519 and #15649.
- August 2018: (merged)#14047 is opened by jonasschnelli. Adds
CKey::Negate()
andHKDF_HMAC_L32
. - March 2019: (merged)#15512 is opened. Instead of bringing in ChaCha20-Poly1305@openssh, this PR leverages the existing ChaCha20 in Bitcoin Core. The existing implementation is only used as a Deterministic Random Bit Generator (DRBG) for Pseudo-RNG purposes. So all that’s needed to be added is XOR the keystream with the plaintext to product ciphertext. Includes bench tests + unit tests.
- March 2019: (merged)#15519 by jonasschnelli is opened. It adds a Poly1305 keyed-hash/message authentication code implementation and bench+unit tests.
- March 2019: (merged)#15649 by jonasschnelli is opened. It combines the Chacha20 implementation from #15512 and Poly1305 implementation from #15519 into the ChaCha20-Poly1305@bitcoin AEAD cipher suite. This is the PR that optimizes the cipher suite for small messages like Bitcoin p2p messages by allowing for less than 3 rounds (which is the lower limit in openssh’s implementation). Includes bench + unit tests.
- March 2020: (closed; superseded)#18242 by jonasschnelli implements
V2TransportSerializer
andV2TransportDeserializer
used in tests only for now- June 2020: ariard hosts PR review club on #18242 - this was before #18242 was put behind the changes to the AEAD in #20962. It will now be rebased after #20962 lands.
- Superseded by #23233 by dhruv
- January 2021: (OPEN)#20962 by jonasschnelli is opened to improve the AEAD per the suggestion by sipa from Nov 2020.
- August 2021: dhruv inherits the PR.
- October 2021: (OPEN)#23233 by dhruv revives #18242. Depends on #20962. Implements
V2TransportSerializer
andV2TransportDeserializer
used in tests only for now.
libsecp256k1:
- September 2021: (OPEN)#979 by sipa implements jacobi symbols to find quadratic residues.
- September 2021: (OPEN)#982 by sipa used #979 and implements an Elligator-squared module.
Refactors to existing code pre-requisite for BIP324 v2 protocol introduction:
- May 2019: (closed)#14046 by jonasschnelli refactors message parsing in
CNetMessage
. This PR was deemed to be taking too complex an approach by makingCNetMessage
an abstract class. It was replaced with #16202. - June 2019: (merged)#16202 by jonasschnelli refactors
net.{h, cpp}
andnet_processing.{h, cpp}
so that sending any message can be independent of the transport used to send it. Because BIP324 is specifically opportunistic in its encryption, some of the node’s connections with be v1, and some v2. Most code should not have to worry about that. The network adapter pattern is used here to rename/introduceV1TransportDeserializer
- August 2019: (merged)#16562 by jonasschnelli continues the refactor from #16202 and creates a
V1TransportSerializer
presumably paving the way for a v2 serializer.
Supporting Pull Requests:
- December 2019 (merged) #17771 by practicalswift is opened. Adds fuzz tests for
V1TransportDeserializer
introduced in #16202 by jonasschnelli. - June 2020: (merged)#19296 by practicalswift is opened. Adds fuzz tests for ChaCha20 as well as Poly1305 amongst other cryptography tools in Bitcoin Core.
- August 2021: (OPEN)#22704 by stratospher is opened. Differential fuzzing bitcoin core ChaCha20 against the original DJB implementation.
- October 2021: (OPEN)#23322 by prakash1512 is opened. Differential fuzzing bitcoin core Poly1305 against Floodyberry’s public domain implementation.
Things I am looking to understand
- TODO: “It should be noted that AEADs, such as ChaCha20-Poly1305, are not intended to hide the lengths of plaintexts. When this document speaks of side-channel attacks, it is not considering traffic analysis, but rather timing and cache side-channels.” at https://tools.ietf.org/html/rfc7905
- Why send raw 32 bytes to indicate v2 rather than a new v1 message to do key exchange and allow for the recipient to accept/reject the encrypted connection request?
- Why is observability enough for MITM attacks?
- Why does the public keys being odd help us?
- BIP324 mentions that the peers must immediately discard the private/pub key pair once the shared secret is established. Why is this?
- How will we do this with seeder nodes? The public key cannot be ephemeral by definition. Can we just re-key on the first leg and the seeder not destroy it’s permanent private key?
- Need to think through re-keying the public key for the seeder.
- Regarding two cipher suite instances:
- If you want to hide the length of the payload, don’t you have to pad the payload? Does encrypting the length alone help? I feel like I am missing something.
- Based on the test vectors in crypto_tests.cpp for the AEAD, the 3-byte length field seems to be the len(encrypted length) + len(encrypted payload). While this is ok, most protocols would just encrypt the size of the following payload. Why did we make this different choice?