outernet – Daniel Estévez

Othernet’s open source open-ondd receiver

Back in 2016, I became interested in Othernet‘s satellite filecasting service, which back then was called Outernet. Outernet was a start up company that offered a free global satellite service with which some internet contents such as weather updates or Wikipedia pages were broadcast. The main goal of the company was making this content available in developing countries, remote locations, and in general, in places that couldn’t afford an internet connection. Outernet started being popular with Amateur radio operators, hobbyists and makers, who saw it as an interesting project to set up at home for fun.

Most of the Outernet’s software stack was open source, and their receivers were some kind of embedded Linux system. However, the key parts of the receiver, which were the demodulator and the implementation of the filecasting protocol, were closed source and the protocols and specs they used were not documented publicly. Unhappy with this, I wrote in their forums asking if these software pieces could be open sourced. After receiving a negative reply, I decided to try to reverse-engineer all this, with the goal of writing some documentation and producing an open-source alternative implementation.

Testing LDPC code erasure decoding performance

In my previous post I talked about the RFC5170 LDPC codes used in Outernet. There I explained in some detail the pseudorandom construction of the LDPC codes and the simple erasure decoding algorithm used both in free-outernet and in the official closed-source receiver.

The Outernet LDPC codes follow what I call the “identity scheme”. This is different from the staircase and triangle schemes introduced in the RFC. The identity scheme already appeared in the literature, but it did not make it into the RFC. See, for instance, the report by Roca and Neumann Design, Evaluation and Comparison of Four Large Block FEC Codecs, LDPC, LDGM, LDGM Staircase and LDGM Triangle, plus a Reed-Solomon Small Block FEC Codec, especially Section 2, where it is called “LDGM”.

I also commented that erasure decoding for an LDPC code (or any other linear code) amounts to solving a linear system. This can be done using any algebraic method, such as Gaussian elimination. However, the simple decoding algorithm used in Outernet is as follows: try to find an equation with only one unknown, solve for that unknown, and repeat until the system is solved. Clearly this algorithm can fail even if the system can be solved (see my previous post for some examples and formal results). I will refer to this algorithm as iterative decoding, as it is done in the RFC.

With these two things in mind, I wondered about the performance of the LDPC codes used in Outernet and the iterative decoding algorithm. I’ve done some simulations and here I present my results.

Outernet LDPC code revisited

I have been preparing the slides for my future talk about reverse-engineering Outernet at FAQin 2018. While doing this, I have been re-reading some of the material about the work done on LDPC code and FEC in Outernet by George Hopkins in January 2017. One of the things I didn’t do back then was to read carefully the LDPC decoding function implemented by George.

In my post I explained that the LDPC code used in Outernet followed RFC5170, and I wondered whether it used the staircase scheme or the triangle scheme. I also commented that erasure decoding with an LDPC code (or any other linear code, actually) was mathematically equivalent to solving a linear system where the unknowns correspond to the erased symbols. I observed that the decoding function looked very different from this mathematical procedure, but should do more or less the same thing. Now I have read the decoder implementation carefully and I have the answer to both questions.

free-outernet gets LDPC decoding

In my previous post I talked about some small updates made by George Hopkins to my free-outernet project. In fact, George has been reverse engineering the ondd binary quite in depth and he has been able to reverse engineer the LDPC code which is used for file FEC. This solves a long-standing issue of free-outernet. Formerly, LDPC decoding was not implemented, so to recover a file successfully all the file blocks had to be received correctly. Now, with LDPC decoding the file can be recovered even if some of the file blocks are lost. Thus, the performance of free-outernet in this aspect should now be the same as the performance of the closed source ondd binary included in the official Outernet receiver. Many thanks to George, as this is a substantial improvement of free-outernet. Here I describe the latest changes made by George in free-outernet.

Updated format for Outernet LDP protocol

After my talk in 33C3, George Hopkins has been doing some further reverse engineering of the Outernet protocols. By looking at the ondd binary, he has discovered that the length of the LDP header is not 6 bytes, but rather 4 bytes. He has sent a pull request with these changes. I have already merged it into free-outernet.

Thus, the last field of the LDP header as described in my previous post, which was formerly known as “B field”, gets now included in the payload of the package. The former “A field” is now the only field that identifies the port or service, and therefore it is now known as “type”. For file and FEC blocks, this change is small, because the former “B field” was always 0, so it gets incorporated into the “file id” field of the payload, which is now 24 bits long instead of 16 bits. For time packets it provides a nice way to interpret the packet as a variable record field structure. This interpretation now explains all the contents of the time packet.

After reviewing and merging George’s changes and while I still have these details fresh in my mind, I have updated the slides I used in my 33C3 talk to reflect these changes. These are the updated slides.

Reverse engineering Outernet talk in 33C3

If you’ve being following me on Twitter lately, you should already know that I’ve being giving a talk this morning at 33C3 about my work reverse engineering Outernet. In cased you missed the talk, the recording and the slides are already up. Many thanks for the organization for their wonderful recordings and streamings of the talks and for keeping everything running smoothly.

Reverse-engineering Outernet in GNU Radio blog

I have published a post in the GNU Radio blog about my reverse engineered GNU Radio Outernet receiver gr-outernet. I cover more or less the same information as in a previous post in this blog, but I include lots of screenshots. Many thanks to Ben Hilburn and Johnathan Corgan for contacting me to write this post in the GNU Radio blog and for their useful suggestions.

Head over to the GNU Radio blog and read the post: Reverse-engineering Outernet.

Reverse engineering Outernet: time and file services

In my last two posts, I’ve being talking about my reverse engineering efforts with the Outernet signal and I’ve described the modulation, coding and framing and the L3 and L4 network protocols used in Outernet. This post is the last in this series. Here I talk about how the time and file services work. Recall that a Free Software implementation of an Outernet receiver based on these descriptions is now available at gr-outernet and free-outernet.

Reverse engineering Outernet: L3 and L4 protocols

This is a follow-up to my last post, where I talked about my efforts to reverse engineer the protocols used in the Outernet L-band signal. Here I will describe the L3 and L4 protocols that are used in Outernet.

This description is solely based upon my reverse engineering efforts. As there is no documentation available for this protocols, I get to name them as I like. Also, I’ll describe the protocols just from how they appear to work. Probably the developers at Outernet had something a bit different in mind. In any case, my understanding of how the protocols work seems quite good, as I have now a functional file receiver called free-outernet. In my next post I’ll talk about how the Outernet time service and file service work.

Reverse engineering Outernet: modulation, coding and framing

Outernet is a company whose goal is to ease worldwide access to internet content. They aim to provide a downlink of selected internet content via geostationary satellites. Currently, they provide data streams from three Inmarsat satellites on the L-band (roughly around 1.5GHz). This gives them almost worldwide coverage. The downlink bitrate is about 2kbps or 20MB of content per day.

The downlink is used to stream files, mostly of educational or informational content, and recently it also streams some APRS data. As this is a new radio technology to play with, it is starting to get the attention of some Amateur Radio operators and other tech-savvy people.

Most of the Outernet software is open-source, except for some key parts of the receiver, which are closed-source and distributed as freeware binaries only. The details of the format of the signal are not publicly known, so the only way to receive the content is to use the Outernet closed-source binaries. Why Outernet has decided to do this escapes me. I find that this is contrary to the principles of broadcasting internet content. The protocol specifications should be public. Also, as an Amateur Radio operator, I find that it is not acceptable to work with a black box receiver of which I can’t know what kind of signal receives and how it does it. Indeed, the Amateur Radio spirit is quite related in some aspects to the Free Software movement philosophy.

For this reason, I have decided to reverse engineer the Outernet signal and protocol with the goal of publishing the details and building an open-source receiver. During the last few days, I’ve managed to reverse engineer all the specifications of the modulation, coding and framing. I’ve being posting all the development updates to my Twitter account. I’ve built a GNUradio Outernet receiver that is able to get Outernet frames from the L-band signal. The protocols used in these frames is still unknown, so there is still much reverse engineering work to do.