Blockstream Satellite: decoding Bitcoin transactions

In my previous post I wrote about the protocols used by Blockstream Satellite. This was motivated by a challenge in GRCon22’s CTF. In that challenge, muad’dib sent the flag as a Blockstream API message and recorded the Blockstream Satellite DVB-S2 downlink as the message was broadcast. The recording was used as the IQ file for the challenge.

In my post, I gave a look at how all the protocol stack for the Blockstream API works: DVB-S2, MPE, IPv4, UDP, plus a custom protocol that supports fragmentation and application-level FEC. However, I didn’t give any details about how the protocols used to broadcast the Bitcoin blockchain work. This runs on another UDP port, independently of the Blockstream API. At that time I didn’t understand much about it, even though during the CTF I was trying to search for the flag in a Bitcoin transaction and looking at the source code of bitcoinsatellite to try to figure out how it worked.

After my previous post, Igor Freire commented some details of the FEC used in bitcoinsatellite. This is quite interesting by itself. Two FEC libraries by Chris Taylor are used: the Wirehair O(N) fountain code for larger blocks, and the CM256 MDS code based on Cauchy matrices over GF(256) (this is very similar to Reed-Solomon used as erasure coding). This motivated me to continue studying how all this works.

Now I have been able decode the Bitcoin transactions in the CTF recording. These don’t use any FEC, since transactions are small. I believe that there aren’t any blocks fully contained in the 35 second recording, so to see how the FEC codes work (which could be quite interesting) I would need a longer recording.

In this post I’ll show how to decode the Bitcoin transaction in Blockstream Satellite. The materials can be found in this repository.

Blockstream Satellite transmits the Bitcoin blockchain and transactions using a variant of the Bitcoin FIBRE protocol. UDP port 4434 is used to send this broadcast data into bitcoinsatellite, which is a fork of Bitcoin Core that adds support the FIBRE protocol.

The structure of the UDP messages is defined in udpnet.h in bitcoinsatellite. The message begins with a header that contains the following fields:

• Checksum 1. 8 bytes.
• Checksum 2. 8 bytes.
• Message type. 1 byte. Its possible values are given by the enum UDPMessageType.

The checksum 1 and checksum 2 fields contain the 16 bytes given by the computation of the Poly1305 hash over the whole UDP message, except for the checksum fields (see the FillChecksum() function). The 32 byte key for Poly1305, which is called magic in the source, apparently comes from a hash of the UDP endpoints (see AddConnectionFromString()).

I don’t understand the role that this checksum plays. When Poly1305 is used as a one-time HMAC, the key should only be known to the sender and receiver, and the same key should not be reused for two messages. If these rules are broken, then anyone can generate messages with valid signatures. In bitcoinsatellite it seems that the key is static and well-known to everyone, so apparently the Poly1305 hash doesn’t really give any protection to the integrity and authenticity of the messages.

All the message payload, including the message type field is scrambled by XORing it with the 8-byte sequence contained in the checksum 1 field. According to Igor Freire, this was used in FIBRE (which ran over the Internet) to prevent detection of the protocol by firewalls. This is what was giving me troubles during the CTF, because I never saw this part of the code, and was trying to make some sense of the binary data without realizing that I should descramble first.

After descrambling the payload of the UDP messages, we can read the message type field. In the data I have extracted from the CTF recording, there are the following types of messages:

• BLOCK_CONTENTS. 3171 messages.
• BLOCK_HEADER_AND_TXIDS. 101 messages.
• TX_CONTENTS. 66 messages.

In this post we will only look at the TX_CONTENTS messages, which contain Bitcoin transactions.

After the message type field, a TX_CONTENTS message has the following fields (see the struct UDPFecMessage). The same header is used for other messages that include FEC, such as the blocks messages.

• Hash prefix. 8 bytes. For blocks this contains the 8 least significant bytes of the block hash. For a transaction, it contains the first 8 bytes of the transaction. The only purpose of this value is to contain an unique ID for defragmentation.
• Object length. 4 byte little-endian. This gives the size of the FEC-encoded data in bytes.
• Chunk ID. 3 byte little-endian. This field contains a fragment counter.

The data is fragmented in chunks of 1152 bytes (except for the last chunk, which may be shorter). Using the chunk ID field, it is possible to do erasure decoding if the data uses FEC. The Bitcoin transactions do not use FEC, however, so it is necessary to receive and concatenate all the chunks in a transaction, although most transactions are quite small and fit inside a single chunk. In fact, in the CTF recording there are 58 transactions that fit in 1 chunk, one transaction that fits in 2 chunks, and one transaction that fits in 6 chunks.

To defragment the transactions I’m simply classifying the messages by hash prefix, sorting by chunk ID, and concatenating all the chunks (which follow the chunk ID field). (Note that bitcoinsatellite does not support defragmenting different hash prefixes in parallel). I have observed that when doing this the resulting size of the data is usually a few bytes (0 to 7 bytes) longer than what indicated by the object length field. I think that bitcoinsatellite doesn’t care about these extra bytes, but I wonder where they come from. They have different values, and do not seem to contain any obvious padding pattern.

After we have defragmented the chunks for each hash prefix, we obtain the Bitcoin transactions. There are several resources that explain how Bitcoin raw (binary) transactions are encoded, and online tools to decode them into human readable format. However, the transactions that we have here are in compressed format. From what I’ve found, this is a somewhat obscure format that is only used to send transactions over the network. It is implemented in compressor.cpp. Here “compressed” is used in the sense of serialization using the minimum number of bits possible for each of the fields, not in the sense of using a compression algorithm.

I haven’t found any tools that would allow me to decompress these compressed transactions, so I’ve made my own. This is a very simple C++ executable that uses the CTxCompressor class to decompress a transaction. The implementation follows what the decompression code in udprelay.cpp does. The first byte of the compressed transaction contains the codec version (which is always 1 in the transactions I’ve seen), and the remaining bytes can be fed into the CTxCompressor as a stream to deserialize the compressed transaction into a CTransactionRef. This can then be serialized back as a raw transaction.

My decompressor executable expects the codec version as a command line argument and reads the rest of the compressed transaction data in hex through the standard input. It outputs the uncompressed transaction in hex.

The decompressor links against some of the static libraries built by bitcoinsatellite (or Bitcoin Core), so to build it it is necessary to build bitcoinsatellite from source and use the BITCOINSATELLITE environment variable to indicate the path of the bitcoinsatellite source when running make to build the decompressor.

The decompressor is called for each of the transactions that we have defragmented. The output of the decompressor can then be fed to bitcoin-tx -json (or alternatively to an online tool) to obtain a human readable version of the transaction. The nice thing about using bitcoin-tx instead of an online tool is that the online tools usually “cheat” and look in the blockchain to show us more data related to the transaction. On the other hand, bitcoin-tx only shows the data that is actually contained in the transaction.

As an example, the first transaction in the recording contains the following compressed data (the first byte is the codec version).

010335c8cd50eeaaf55042426eb1669425167af300342ad0496e0c309f5254b56ac5130860f531396979a357f15904be10ec1acf72bb359ca90a7cb9f2df4999beff04aabc12d975e08ed7739c96df9fee71bbce97498bd517787fd97b851c03f06037ee1c041eeb4a77757e252d281bcabc163cdc52d76d2be5547902214d84a9e3bb822904f6349bc79dbec17997c7118e44266b9fa8fec7f881270120ccc1f192df48568ae82331c9b16660e8d0900ac15d

This is decompressed to produce

0100000001c8cd50eeaaf55042426eb1669425167af300342ad0496e0c309f5254b56ac513010000006b483045022100f531396979a357f15904be10ec1acf72bb359ca90a7cb9f2df4999beff04aabc022012d975e08ed7739c96df9fee71bbce97498bd517787fd97b851c03f06037ee1c012103041eeb4a77757e252d281bcabc163cdc52d76d2be5547902214d84a9e3bb8229ffffffff021009050000000000160014f6349bc79dbec17997c7118e44266b9fa8fec7f8b5030000000000001976a91420ccc1f192df48568ae82331c9b16660e8d0900a88ac00000000

The corresponding output of bitcoin-tx -json is

{'txid': '9a1bb9abf0b735e8ccffc823abea56b1db21c36493670009aee6e919d5c09b78',
 'hash': '9a1bb9abf0b735e8ccffc823abea56b1db21c36493670009aee6e919d5c09b78',
 'version': 1,
 'size': 223,
 'vsize': 223,
 'weight': 892,
 'locktime': 0,
 'vin': [{'txid': '13c56ab554529f300c6e49d02a3400f37a16259466b16e424250f5aaee50cdc8',
   'vout': 1,
   'scriptSig': {'asm': '3045022100f531396979a357f15904be10ec1acf72bb359ca90a7cb9f2df4999beff04aabc022012d975e08ed7739c96df9fee71bbce97498bd517787fd97b851c03f06037ee1c[ALL] 03041eeb4a77757e252d281bcabc163cdc52d76d2be5547902214d84a9e3bb8229',
    'hex': '483045022100f531396979a357f15904be10ec1acf72bb359ca90a7cb9f2df4999beff04aabc022012d975e08ed7739c96df9fee71bbce97498bd517787fd97b851c03f06037ee1c012103041eeb4a77757e252d281bcabc163cdc52d76d2be5547902214d84a9e3bb8229'},
   'sequence': 4294967295}],
 'vout': [{'value': 0.0033,
   'n': 0,
   'scriptPubKey': {'asm': '0 f6349bc79dbec17997c7118e44266b9fa8fec7f8',
    'desc': 'addr(bc1q7c6fh3uahmqhn978zx8ygfntn750a3lc8l2gfr)#wrj3r36g',
    'hex': '0014f6349bc79dbec17997c7118e44266b9fa8fec7f8',
    'address': 'bc1q7c6fh3uahmqhn978zx8ygfntn750a3lc8l2gfr',
    'type': 'witness_v0_keyhash'}},
  {'value': 9.49e-06,
   'n': 1,
   'scriptPubKey': {'asm': 'OP_DUP OP_HASH160 20ccc1f192df48568ae82331c9b16660e8d0900a OP_EQUALVERIFY OP_CHECKSIG',
    'desc': 'addr(13zRuwqgCsEmGNsFDyWhYaSoxva6KdUnaU)#0l3w8m74',
    'hex': '76a91420ccc1f192df48568ae82331c9b16660e8d0900a88ac',
    'address': '13zRuwqgCsEmGNsFDyWhYaSoxva6KdUnaU',
    'type': 'pubkeyhash'}}],
 'hex': '0100000001c8cd50eeaaf55042426eb1669425167af300342ad0496e0c309f5254b56ac513010000006b483045022100f531396979a357f15904be10ec1acf72bb359ca90a7cb9f2df4999beff04aabc022012d975e08ed7739c96df9fee71bbce97498bd517787fd97b851c03f06037ee1c012103041eeb4a77757e252d281bcabc163cdc52d76d2be5547902214d84a9e3bb8229ffffffff021009050000000000160014f6349bc79dbec17997c7118e44266b9fa8fec7f8b5030000000000001976a91420ccc1f192df48568ae82331c9b16660e8d0900a88ac00000000'}

Now that we have the transaction ID, we can use any of several online tools to look up this transaction in the blockchain. For instance, here is what blockchain.com shows. We see that the transaction was received on 2022-09-05 07:09 UTC (note that blockchain.com displays the timestamps in local time). This is an interesting piece of data, because previously I hadn’t found anything in the data itself that would allow us to know when muad’dib did the CTF recording. Since transactions are broadcast by Blockstream Satellite more or less in real time, this gives us a timestamp that is probably accurate to a couple minutes. In fact, the recording filename contains 2022-09-05_030938, which I guess is an EDT (UTC-4) timestamp. We can also see that the transaction was eventually included in block 752704, which was mined at 07:18:48 UTC.

To check that I have decoded all the transactions correctly and that there are no weird things, I have downloaded some data for the all transactions in 2022-09-05 from the blockchair.com transactions database. This is a simple plaintext file containing a table with some details of each transaction. What I do to check the transactions is to get their ID using bitcoin-tx -json, and then search the ID in the blockchair.com file and get the corresponding block height. Doing this, I find that 41 transactions ended up in block 752704, 13 transactions ended up in block 752705, and 6 transactions ended up in block 752706. Additionally, there is one transaction that doesn’t appear in the blockchair.com file.

This missing transaction is the following:

{'txid': '5598ef82876f3f227ae84f587fc67267364f04ce8721248604e8fe680d9b9f27',
  'hash': 'b9d3a91cd9c46346316ecb187d1c7eb05155c0fd9848401bde8578e48c669b20',
  'version': 2,
  'size': 191,
  'vsize': 110,
  'weight': 437,
  'locktime': 0,
  'vin': [{'txid': '699f16147878c8dfefe88921cdcef72cacd24a212b101e09099342c628baaf43',
    'vout': 62,
    'scriptSig': {'asm': '', 'hex': ''},
    'txinwitness': ['304402207c45cef50619d9fa94a4ab5fe17ecd30f65f274820808b8b3e4b5aeafb71003e02202791dd28850f1f3be68a1a3dbb736245c64f1c171012189766b03d2fbc376b0301',
     '0377bae134cbdacc9c40b381eacff70e45a68f3f45eac8648bb3381c7ba9595f7c'],
    'sequence': 4294967293}],
  'vout': [{'value': 0.89339695,
    'n': 0,
    'scriptPubKey': {'asm': '0 7749f9e8a4a42d27327ac6406e24491e406ae24f',
     'desc': 'addr(bc1qwayln69y5skjwvn6ceqxufzfreqx4cj0znkntg)#6vlhur7l',
     'hex': '00147749f9e8a4a42d27327ac6406e24491e406ae24f',
     'address': 'bc1qwayln69y5skjwvn6ceqxufzfreqx4cj0znkntg',
     'type': 'witness_v0_keyhash'}}],
  'hex': '0200000000010143afba28c6429309091e102b214ad2ac2cf7cecd2189e8efdfc8787814169f693e00000000fdffffff012f375305000000001600147749f9e8a4a42d27327ac6406e24491e406ae24f0247304402207c45cef50619d9fa94a4ab5fe17ecd30f65f274820808b8b3e4b5aeafb71003e02202791dd28850f1f3be68a1a3dbb736245c64f1c171012189766b03d2fbc376b0301210377bae134cbdacc9c40b381eacff70e45a68f3f45eac8648bb3381c7ba9595f7c00000000'}

Its ID doesn’t appear in any of the online blockchain lookup tools, so this transaction never made it into the blockchain. With the help of the online transaction decoder tool we find that the input for this transaction is address bc1qpztxcfh7y4r58s0y0e97h8tt5p9n5v6h8eu39p. (Here the online tool is helpful because otherwise it is a bother to go to the input transaction and look for vout 62).

Looking up this address, we find transaction 6ef6e2eb20ad189ffc95adea02af21aca39a58ce42540960b1fd6b340d6c792c. We can fetch the raw data for this transaction using this tool, obtaining

0200000000010143afba28c6429309091e102b214ad2ac2cf7cecd2189e8efdfc8787814169f693e00000000fdffffff01f815530500000000160014f6d6ef8afa5ba9bfe283851f4550021bb4cb1ed202473044022031deacf5563a12f42051d8c899eece07a687a9abbc08bdad4b8cc095ecf0dea402201f3abc681e37c49f34d0b04d1ca226c344a48cb548fc7d6087cb4957e3470ef301210377bae134cbdacc9c40b381eacff70e45a68f3f45eac8648bb3381c7ba9595f7c00000000

The bitcoin-tx -json decodes the following:

{
    "txid": "6ef6e2eb20ad189ffc95adea02af21aca39a58ce42540960b1fd6b340d6c792c",
    "hash": "aaf733752a8cd64fe7a8564c9fb4875cb456f20f1f7f4e4b426b44c128d13a5d",
    "version": 2,
    "size": 191,
    "vsize": 110,
    "weight": 437,
    "locktime": 0,
    "vin": [
        {
            "txid": "699f16147878c8dfefe88921cdcef72cacd24a212b101e09099342c628baaf43",
            "vout": 62,
            "scriptSig": {
                "asm": "",
                "hex": ""
            },
            "txinwitness": [
                "3044022031deacf5563a12f42051d8c899eece07a687a9abbc08bdad4b8cc095ecf0dea402201f3abc681e37c49f34d0b04d1ca226c344a48cb548fc7d6087cb4957e3470ef301",
                "0377bae134cbdacc9c40b381eacff70e45a68f3f45eac8648bb3381c7ba9595f7c"
            ],
            "sequence": 4294967293
        }
    ],
    "vout": [
        {
            "value": 0.89331192,
            "n": 0,
            "scriptPubKey": {
                "asm": "0 f6d6ef8afa5ba9bfe283851f4550021bb4cb1ed2",
                "desc": "addr(bc1q7mtwlzh6tw5mlc5rs50525qzrw6vk8kj685aef)#udae6tx9",
                "hex": "0014f6d6ef8afa5ba9bfe283851f4550021bb4cb1ed2",
                "address": "bc1q7mtwlzh6tw5mlc5rs50525qzrw6vk8kj685aef",
                "type": "witness_v0_keyhash"
            }
        }
    ],
    "hex": "0200000000010143afba28c6429309091e102b214ad2ac2cf7cecd2189e8efdfc8787814169f693e00000000fdffffff01f815530500000000160014f6d6ef8afa5ba9bfe283851f4550021bb4cb1ed202473044022031deacf5563a12f42051d8c899eece07a687a9abbc08bdad4b8cc095ecf0dea402201f3abc681e37c49f34d0b04d1ca226c344a48cb548fc7d6087cb4957e3470ef301210377bae134cbdacc9c40b381eacff70e45a68f3f45eac8648bb3381c7ba9595f7c00000000"
}

Note that this has exactly the same input as our missing transaction. Additionally, the transaction that did end up in the blockchain has 8.5e-5 BTC more fees. So what happened is probably that the missing transaction was sent first, and then sometime later the transaction that ended up in the blockchain was sent with higher fees to try to speed up the confirmation, so the first transaction simply got discarded. The second transaction was received at 07:18 UTC by blockchain.com, and the first transaction should have been received around 07:09 UTC. Interestingly, the two transactions used a different output address.

In this data there was also a rather long transaction that was fragmented in 6 chunks. Apparently this has also been decoded correctly, showing that the defragmentation works. Its ID is 75e3d4bf976e7d0ca03c3ee33e7ac43ea324cf177178288adab4f550902c10dc. We can see that it is a transaction with many inputs, which explains its large size.

Part of my motivation for looking at these Bitcoin transactions in details comes because when I tried to solve the CTF challenge I didn’t know that Blockstream API messages existed. Therefore, the way that I imagined that the challenge would have been set up was to include the flag in a Bitcoin transaction. I knew that it was possible to add short messages in Bitcoin transactions.

I have even tested this when writing this post by inserting the message flag{this is a BTC flag} in a transaction in the Bitcoin testnet. To do this, I got some coins from the faucet, and used

bitcoin-tx -testnet -create -json \
    load=privatekeys:keys.json load=prevtxs:prev.json \
    in=3da53fdc9abaf77bdb7d8160e2b1a919cce00ea317c1fd690a0a10527083aa06:1 \
    outaddr=0.00009:tb1ql7w62elx9ucw4pj5lgw4l028hmuw80sndtntxt \
    outdata=666c61677b7468697320697320612042544320666c61677d \
    sign=ALL

The contents of the files keys.json and prev.json were respectively

{"tb1q0xg20735a98yzq3s8f84zmtpdyekytn3nnfgy3": "cQGQgNQqhBgu4nX1rUqNWQ8HqpeQxv9q1zwwggBioYs2fhBzMj7u"}

and

[{
"txid":"3da53fdc9abaf77bdb7d8160e2b1a919cce00ea317c1fd690a0a10527083aa06",
"vout":1,
"scriptPubKey":"00147990a7fa34e94e4102303a4f516d616933622e71",
"amount":0.0001
}]

To get a private key and address, I created a new wallet with Electrum and exported the private key of one of the addresses. The outdata used in the transaction was obtained with

printf "flag{this is a BTC flag}" | xxd -ps

I broadcast the transaction to the testnet with blockstream.info, which conveniently gives a preview of the transaction and a warning if the fees are too low .

I find that the idea to do this kind of CTF challenge with a transaction instead of an API message is interesting. Muad’dib paid a small amount of money to Blockstream to get the flag broadcast as an API message. I think it should be possible to put the flag in a transaction that has very small fees, so that perhaps it never gets mined and gets dropped (in which case sending the flag would effectively be free, and otherwise we would only pay the very small fees we’ve set). Note that the fees need to be high enough so that the transaction makes it to the mempool and gets shared by the servers. I don’t know the details about fees, so I’m not sure if this trick would be realistically possible, and if it would cost less than the API message if it isn’t free.

I have updated the Github repository to include the tool to decompress transactions and a new version of the Jupyter notebook.