Reverse engineering Outernet: time and file services

In my last two posts, I’ve being talking about my reverse engineering efforts with the Outernet signal and I’ve described the modulation, coding and framing and the L3 and L4 network protocols used in Outernet. This post is the last in this series. Here I talk about how the time and file services work. Recall that a Free Software implementation of an Outernet receiver based on these descriptions is now available at gr-outernet and free-outernet.

Currently, Outernet provides only two services: the time service and the file service. The purpose of the time service is to update the clock of the receivers, and the purpose of the file service is to broadcast files, which is Outernet’s main service.

Time service

The Outernet time service is used to update the clock on the Outernet receivers, since these receivers are designed to run without internet connectivity and they usually do not have a real time clock. The time service broadcasts a time packet approximately every minute. This packet contains a Unix timestamp (as a big-endian 32bit integer), and the Outernet receiver software uses this timestamp to set the system clock.

These are the contents of an OP packet from the time service, as broadcast from the Americas Inmarsat satellite. I have removed the Ethernet headers and the trailing zeros.

0000   00 1c 3c 00 00 00 81 00 00 18 01 04 6f 64 63 32
0010   02 08 00 00 00 00 58 02 63 5b 19 da 95 3d

As I said in the previous post, the time packets are LDP packets with an A field of 0x8100 and a B field of 0x0104. The first bytes of the LDP payload are 0x6f 0x64 0x63 0x32, which is ASCII for odc2. As we will see, in the time packets broadcast from the Europe/Africa satellite, odc3 is used instead. This field is used to identify which satellite/groundstation is broadcasting, since there is no information about this on the other packets other than the source MAC of the Ethernet frames, which may change if they change equipment on the groundstation.

The letters “odc” stand for Outernet Datacasting (I’ll explain below why I know this). In fact, currently there is HTTP access to odc2.outernet.is, odc3.outernet.is and odc4.outernet.is. They all show the login page of an M7 Satellite Modem, so it seems that Outernet has the satellite modems at its three groundstations exposed online (a terrible idea if you ask me). I have just noticed this while writing this post. I don’t know if knowing that the satellite modem used is the M7 would have helped me to reverse engineer the modulation, coding and framing. In the M7 datasheet you can see that it supports much more advanced stuff that the basic schemes that are used for Outernet. We also see that one of the operating modes of the modem is “Ethernet”, so now we see why the Outernet frames are Ethernet frames. The frames are generated in a computer, which sends them to the M7 modem by Ethernet (as broadcast frames) and the modem just retransmits these frames.

It’s also fun to try to geolocate the IPs for odc2, odc3 and odc4. I’m using geoip.co.uk for this. Currently odc2 has the IP 216.129.171.61, which is geolocated to Toronto, odc3 has the IP 212.165.126.66, which is geolocated to Amsterdam and odc4 has the IP 123.100.88.137, which is geolocated to Ketu Bay, New Zealand. Other tools may give slightly different results, since IP geolocation is not an exact science.

I don’t know what are the next two bytes, which are always 0x02 0x08 and why the next 4 bytes are zeroed out. The next four bytes are the Unix timestamp. In this case, 0x5802635b corresponds to 15 Oct 2016 17:11:55 UTC. The last four bytes are the LDP checksum.

This is a time packet as broadcast from the Europe/Africa Inmarsat satellite.

0000   00 1c 3c 00 00 00 81 00 00 18 01 04 6f 64 63 33
0010   02 08 00 00 00 00 58 02 5d 9d f8 16 9e 39

The only difference is that odc3 is used instead of odc2. I haven’t seen any packet from the Asia/Pacific satellite, but I guess that it uses odc4.

Note that this time service is not very good in comparison with other usual time services such as GPS or NTP. The resolution is only one second and the round trip time to geostationary orbit is not accounted for. However, it’s useful for its intended application, which is to keep the clocks of the receivers reasonably on time.

File service

The file service is Outernet’s main service and is used to broadcast files. It uses three types of packets, and each type uses different LDP A and B fields, so they can be processed correctly in the file service client application. The three types of packets are: file announcements, file blocks and FEC blocks.

File announcements

The transmission of a file starts with the transmission of the corresponding file announcement. This is a large LDP packet that usually gets split in 6 OP fragments. The A field used for file announcements is 0x6900 and the B field is 0x0302. The payload of the file announcement LDP packet is essentially an XML (in ASCII) containing the description of the file details. However, the file announcement is signed with an X.509 certificate or something like that. I guess that this is done to prevent spoofing, but I think one it’s unlikely to get spoofing in a satellite application such as this one. One could argue that it is used so that if Outernet manages to sell a lot of receivers another company can’t start to broadcast a compatible service for which Outernet receivers can be used. So, first you have the X.509 thing, which is mostly binary but has some ASCII strings and then you have the XML with the file description.

I haven’t tried to see how X.509 is being used to sign file announcements, since I don’t much interest in this. Outernet’s ondd binary is distributed with a CA certificate, so it surely checks the X.509 certificate signing the file announcement against this CA certificate. The ASCII strings found within the X.509 data are “Illinois”, “Chicago” (the location for the X.509 certificate), “Outernet”, “Datacasting”, the dates of creation and expiration of the certificate (it’s valid from 2015 to 2031) and “odc3.outernet.is” (in packets from the Europe/Africa satellite). Of course all of these are the different fields of the X.509 certificates. Perhaps if one passes the binary info to openssl, it will give something more. It is from this data from where I have guessed that “odc” stands for “Outernet Datacasting” and from where I got the idea to look up odc3.outernet.is in a web browser.

This is an example of an XML file description. It has being formatted for readability. The original doesn’t contain line breaks or indentation.

<?xml version="1.0" encoding="UTF-8"?>
<file>
  <id>2380</id>
  <path>opaks/dad7-Alt-right.html.tbz2</path>
  <hash>aed3e3b58193bdda9af9adb700972cb
         426ca26b336e36c2dfa0175b6e1deb4c8</hash>
  <size>109186</size>
  <block_size>242</block_size>
  <fec>ldpc:k=452,n=543,N1=2,seed=1000</fec>
</file>

This is mostly self explanatory: the id is used to identify the file in file block and FEC block packets, the path is the path were the file should be saved (compressed files are always saved to the opak folder and the closed-source ondd client decompresses them after they have being downloaded correctly), hash is the SHA256 hash of the file, size is the size of the file in bytes, block_size is always 242 and is the size of the blocks that the file is split into (more about this later) and fec contains the specifications for the FEC used for the file.

FEC (Forward Error Correction) allows the receiver to recover the complete file even if some of the blocks are missing, because the signal was not very good at some times or the path to the satellite was interrupted briefly. This is a good idea, because each block of the file is broadcast only once. It seems that all but the smallest files are sent with FEC (in case there is no FEC the fec tag is just missing) and that an LDPC code is always used for FEC. The specifications of the LDPC code vary a lot from file to file and are probably chosen according to file size using some algorithm.

After the file announcement is sent, the file blocks and FEC blocks for the file are sent. Both the file blocks and the FEC blocks are sent in order, but file blocks and FEC blocks are sent in an interleaved manner. For any particular file, there are many more file blocks than FEC blocks, so the satellite spends most of the time sending file blocks. It seems that Outernet only sends one file at a time, although there is no reason why several files can’t be sent at the same time, provided that the file announcements for all of them have being done before sending the corresponding file and FEC blocks. There’s also no reason why blocks need to be sent in order.

File blocks

Each file is split into blocks of 242 bytes. When adding a 4 byte header to one of these blocks and embedding it as the payload of an LDP packet, an OP packet of 262 bytes (which is the Outernet MTU) is produced. This explains the choice of 242 bytes as the block size. The last block of the file is generally smaller than 242 bytes, so it is sent in a smaller LDP packet.

This OP packet is an example of a file block. In fact it is the first block of some file.

0000   01 04 3c 02 00 00 18 00 01 00 00 00 06 d9 00 00 
0010   42 5a 68 39 31 41 59 26 53 59 02 3c e3 ff 01 2b
0020   03 7f ff ff ff ff ff ff ff ff ff ff ff ff ff ff
0030   ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
0040   ff ff ff e3 3c bf 00 e8 e8 75 f2 5d de 77 9e ee
0050   fb 55 aa f2 ad 0f ae 63 5d e9 e7 de ee ee ed bb
0060   e3 5c b2 f3 65 7a c6 79 80 47 be df 3b e3 a7 3a
0070   6d af b7 ab b9 de 6e fa dd 3a fb dd 3b be d9 4e
0080   dd f0 0e ec 75 e6 af 2e f9 f1 6f 9e eb e7 df 3b
0090   ef bc f3 7b ed cd be b7 3b 3b 7d be e7 65 bd 15
00a0   ef be b5 b8 de b2 b1 5b ee fb dd be f7 52 f3 74
00b0   ef 7d dd d6 f7 bb 9b eb 3e ae ef 79 d8 df 4f be
00c0   a6 f6 dd 74 db dd f7 bb ef 9b de b7 cf 5d 5d 7b
00d0   7d de be f6 df 73 cf 35 ee 57 4d 6d 5f 6e ef 77
00e0   ca fa f1 eb 7d cf 7d ef 7b ef 7d f6 fb cc da 2b
00f0   be ce fb df 7b a7 d7 db 1e be 9d 3d 1d dc 77 76
0100   a0 ef ea 02 69 78                              

You can see that file blocks are sent with an LDP A field of 0x1800 and a B field of 0x0000. The payload of the LDP packet starts with the file id as a 16bit integer. Here we have 0x06d9, which stands for id 1753. The next two bytes are the block number. In this case, their value is 0x0000, since this is the first block. Then the 242 byte block starts.

Since this is a bzip2 compressed file, you can see that the first bytes 0x42 0x5a 0x68 are ASCII for BZh, which is the magic number for bzip2 files. After this you have the byte 0x39 an then you have 0x314159, which, if you haven’t noticed, is the first digits of the number \(\pi\) in BCD. I think this has being included in the bzip2 format to aid file carving. I think it’s neat to show this. If you’re interested, you can find what are the other bytes of the bzip2 header in Wikipedia.

There is not much else to say about file blocks. The receiver just has to join all the file blocks in the correct order to recover the file. This is provided it managed to get all the blocks correctly. If it has missed some of the blocks, this brings us to FEC.

FEC blocks

The idea behind FEC is simple: to take a message (in this case the whole file binary data) to be transmitted and produce from it some amount of additional information (sometimes known as FEC data). This is known as FEC encoding. Then the message and the FEC data are sent. Perhaps some parts of the message or FEC data are received with errors (some bits have changed their value) or some parts are missing. Under some conditions, the FEC decoding algorithm manages to recover the whole message even if there are errors or missing parts, by using the extra FEC data. In Outernet, the data is never received with errors, since HDLC frames are checked with their CRC-16CCITT (and LDP frames have an additional 32bit checksum). However, it’s perfectly possible that there are missing file blocks if the signal was not good at times. These missing parts are known as erasures in FEC parlance. FEC is used to try to recover the whole file even in this circumstances.

The FEC data is sent in 242 byte blocks similarly to how files are sent. This is an OP packet with a FEC block.

0000   01 04 3c 02 00 00 ff 00 01 00 00 00 06 fe 01 24 
0010   41 ce b4 5b 28 9d 41 07 f7 b5 5d d8 50 61 ff e0
0020   5f 3a 85 2b 30 de 0c bf 07 04 4d 68 00 c8 00 fe
0030   01 1a 43 a3 5b 94 cd ac 76 d9 e2 69 3d f7 6b dc
0040   42 1e e8 a5 98 f6 28 6d 1b 72 f9 6f 3d fb 5a 63
0050   0b 3d e7 2d 49 52 ce 06 7f 38 e5 65 d6 85 d3 a1
0060   4b 95 8f 90 bc 49 59 46 7a 91 df fe 94 9f 88 0b
0070   fc 25 54 4c 3e 09 ae 44 4b bc b8 e6 27 5d bc 81
0080   31 68 d8 9e f6 40 51 5d 54 3e 0b 55 68 2b 4d a3
0090   91 6c bf c3 ef 2c 89 ef 28 67 3e d0 de f7 c4 91
00a0   fa 1e 58 bd 48 80 53 85 a6 29 2c d4 bb a4 8c 5f
00b0   4d 63 8f 1e 3c 17 39 b9 18 52 ee 05 a8 3f 05 6e
00c0   c1 da 03 b0 6e c9 ce f7 b2 69 ae d1 20 85 79 23
00d0   a9 f0 22 cc 79 fb 92 06 e8 9c 54 df 18 09 a4 57
00e0   a3 e6 ed 76 89 ba 85 f9 9e 11 f4 48 d7 1b 12 20
00f0   ec 50 79 b8 44 91 a2 f4 4d 64 ed 80 e1 8e 53 b4
0100   27 2c 18 c4 54 04

As you can see, FEC blocks are sent in LDP packets with an A field of 0xff00 and a B field of 0x0000. The payload of the LDP packet has the same structure as the file blocks. The first two bytes are the file id, in this case 0x06fe or 1790 and the next two bytes are the block number, in this case 0x0124, or block number 292.

The FEC codes used for Outernet produce FEC data whose size is around 20% of the size of the original file. One thing that I find curious is that all the FEC blocks are 242 bytes long (and not less), so the FEC code is arranged so that it always produces FEC data whose size is a multiple of 242 bytes (it doesn’t seem that some part of the FEC data is clearly padding).

Unfortunately, that’s as much as I can say about FEC. The LDPC codes used for FEC in Outernet are not so easy to reverse engineer for me. There are many different LDPC codes, so it’s hard to guess which are being used, and almost every file uses a different set of parameters for its LDPC code. Also, implementing an LDPC decoder is not so trivial. For this reason, I don’t intend to work in this in the near future, unless someone with more experience on LDPC codes wants to help.

The free-outernet receiver does not implement LDPC decoding. This means that you need to receive all the file blocks to recover a file completely. However, it is perfectly usable if you have a good signal. Also, the smallest files (APRS data, for instance) don’t use FEC at all.

Actual data rate

One thing one can do with this knowledge of the protocols is to compute the actual bandwidth that is available to send files. The file blocks are 242 bytes long, but they are sent in 276 byte Ethernet frames. This means that there is an overhead of 12% for headers. Also, there is an overhead of 20% for FEC data. This works out to a total overhead of 30%. Since the bitrate is 2.1kbps, the actual bandwidth that is used to send files is only 1.47kbps. This amounts to 15876kB of files per day. Here 1kB = 1000B, but keep in mind that 15876kB is only 15.14MB if you use the convention that 1MB = 1024kB, 1kB = 1024B. This is noticeably less that the 20MB per day that Outernet quotes.