Using GSE and DVB-S2 for IP traffic

GSE (Generic Stream Encapsulation) is a protocol used to embed packets of almost any sort into the DVB data link layer. It can be used to send IP (IPv4 and IPv6) packets, Ethernet packets, etc. In my post about Blockstream Satellite, I talked about MPE, which is another way of sending IP traffic inside DVB. However, MPE is based on MPEG TS packets, so it is a far from ideal solution, given the overhead of the TS headers and the relatively small size of TS packets. GSE is a much more lightweight solution, and it’s arguably the best way of sending IP packets inside DVB.

The downside of GSE compared to MPE is that it is not supported by so many devices. Since MPE uses TS packets, it should be supported by mostly any device. The formatting of the TS packets, and thus all of the MPE stack, is handled at the application level. However, GSE is different from a stream of TS packets already the level of BBFRAMEs, so devices that handle this layer need to support GSE.

In this post I show how to set up a DVB-S2 GSE one-way link using the GNU Radio out-of-tree module gr-dvbgse and an SDR for transmission, and a MiniTiouner, Longmynd and some software I’ve written for reception.

The MiniTiouner is a DVB-S2 hardware receiver that is based on a Serit FTS4334 NIM (which uses the STV0910 DVB-S2 demodulator IC) together with a FT2232H that provides a USB2 interface for data and control. It is a very popular device within the Amateur TV community, given its affordable price and large range of supported carrier frequencies, symbol rates, and MODCODs.

The ideas in this post are also applicable to an SDR demodulation approach, which could use gr-dvbs2rx and gr-dvbgse. Using a hardware receiver solution can give some benefits over an SDR receiver, since demodulation and LDPC decoding is computationally expensive, specially at higher symbol rates and in low SNR conditions.

My final goal for this is to do some tests of two-way IP links over the QO-100 WB transponder. I think this would be a rather interesting use of the transponder, since it would open the door to many new ideas. Currently the transponder is used almost exclusively to transmit video, which by all means is good, but not very innovative after the almost 4 years now that the transponder has been in operation.

I have to give huge thanks to Brian Jordan G4EWJ and Evariste Courjard F5OEO for their interest in this project and for running many initial tests that showed that it is possible to use the MiniTiouner to receive GSE (despite the lack of clear and detailed documentation about the STV0910 register settings).

Hardware

For my tests I am using a USRP B205mini as the transmitter, though any SDR capable of supporting the bandwidth required for the symbol rate of interest can be used. This is connected to the BATC MiniTiouner v2.0 through 50 dB of attenuation, some 75Ω TV coax, and a DC block. The USRP and MiniTiouner are connected to the same PC by USB.

Transmitter

For the transmitter, I am using gr-dvbgse. I have taken the example TX flowgraph, changed the symbol rate and MODCOD, and adapted the flowgraph for my USRP. I’m using 1 Msym/s QPSK 1/2 with normal FECFRAMEs and no pilots, since that’s a very straightforward configuration. I’m running the USRP at 2 Msps at a carrier frequency of 750 MHz (which doesn’t really matter, but this is close to the IF frequency for the QO-100 WB transponder when using a standard 9750 MHz LO). I haven’t bothered to set up offset tuning (which is something that the original example flowgraph does). I’ve set a TX gain of 40 dB.

In the IP Packet Source block, I have turned off the “Ping Reply” option, since I don’t want my ping packets to be modified (the README describes how this option works).

I have followed the instructions in the README to set up a TAP network device:

sudo ip tuntap add dev tap0 mode tap
sudo ip link set dev tap0 address 02:00:48:55:4c:4b
sudo ip addr add 44.0.0.1/24 broadcast 44.0.0.255 dev tap0
sudo ip link set tap0 up
sudo ip link set tap0 arp off

Additionally, note that it’s necessary to increase the capabilities of the python3.10 binary (or to run the flowgraph as root):

sudo setcap 'CAP_NET_RAW+eip CAP_NET_ADMIN+eip' /usr/bin/python3.10

This is a potential security hole, so it should be undone after testing with

sudo setcap '' /usr/bin/python3.10

At this moment, we can start the transmitter flowgraph.

Longmynd

To control the MiniTiouner, I am using Longmynd. This is run as

./longmynd -r -1 -i 127.0.0.1 4003 -I 127.0.0.1 \
    4002 750000 1000

The -r -1 option is necessary, because otherwise Longmynd will complain that there is no data and will try to retune when the GSE stream doesn’t carry any packets. We are setting Longmynd to send data to UDP port 4003 on localhost and status information to UDP port 4002. The last two parameters correspond to the carrier frequency and symbol rate in kHz, and should match those used on the transmitter.

Longmynd needs access to the USB device of the MiniTiouner (a device under /dev/bus/usb/), so we either need to use some udev rules or chmod 666 the appropriate device. If Longmynd cannot access the device it will indicate its path, giving an easy way to figure out to which device we need to apply the chmod.

We can display the status information by using

nc -ulp 4002

We should check that the demodulator is locked, which is indicated by the $1,4 line, that the correct MODCOD is detected ($18,4 for QPSK 1/2), that the presence of short FECFRAMEs and pilot symbols (lines $19 and $20) is detected correctly, and that the MER is high (line $12, given in units of 0.1 dB). In my case I get a MER of 25.6 dB.

Brian G4EWJ has found that when the MiniTiouner receivers a generic continuous stream (which is the mode used by GSE), then Longmynd ends up sending BBFRAMEs (including the BBHEADER) to the UDP data port (4003 in this case) fragmented in packets with 510 bytes of payload. There are more details about this in the appendix at the end of the post.

dvb-gse

I have written a Rust command line application called dvb-gse that receives the BBFRAME fragments sent by Longmynd, performs defragmentation to get full BBFRAMEs, handles the GSE protocol (which also needs defragmentation) and sends the IP packets to a TUN device.

The way to install this application is with cargo (Rust’s “package manager”) by running

cargo install dvb-gse

This requires the Rust compiler and cargo to be installed. The easiest way to do this is with rustup.

Before running dvb-gse, it is necessary to set up a TUN device similarly to how we have added the TAP device for gr-dvbgse. It is not necessary to give an IP address to the TUN device, since we will only use it to receive packets.

sudo ip tuntap add dev tun0 mode tun
sudo ip link set tun0 up

Now we can run dvb-gse as follows. This will receive UDP packets on port 4003 and send the IP packets to tun0.

dvb-gse --listen 127.0.0.1:4003 --tun tun0

dvb-gse uses env_logger for logging, so to troubleshoot it is possible to run it with a more detailed logging level such as debug or trace by running it as

RUST_LOG=debug dvb-gse --listen 127.0.0.1:4003 --tun tun0

Testing

Now we can use Wireshark to monitor the packets received by the interface tun0. We can send some IPv4 packets by pinging 44.0.0.2 (recall that we gave the address 44.0.0.1/24 to the tap0 interface).

ping -n 44.0.0.2

We can send some IPv6 packets by pinging a link-local address on the interface tap0:

ping fe80::1%tap0

There will be no reply to these ping packets, since we have set a one-way link only. The packets received by Wireshark can be seen below (click on the image to display it in full size).

We can test streaming something to check that the link works with more realistic traffic than just one ping packet per second. Doing this on a single machine is a bit tricky, because unless we’re careful the packets will be directly delivered by the kernel between applications rather than going through the DVB-S2 link.

I have found that a way to do this test is to use IPv6 link-local addresses. The tun0 interface already has a link-local address given by the kernel (which can be seen with ip address show tun0). In my case, the address is fe80::4f7f:8083:683:69c6/64. With this knowledge, we can instruct cvlc to stream to this IP address on the interface tap0.

cvlc file.mp3 –sout udp:\[fe80::4f7f:8083:683:69c6%tap0\]:8090

Here file.mp3 is a VBR MP3 file that I have lying around. Note that the bitrate of the file to be streamed should be smaller than the bitrate provided by the DVB-S2 link, which in my case (1 Msym/s QPSK 1/2) is slightly below 1 Mbps. This will make VLC stream the file as a TS stream in UDP packets, so the media file should be TS-compatible (FLAC is not allowed, for instance).

I haven’t managed to get either cvlc or mpv to receive these UDP packets. I have resorted to use nc to receive the data, and then pipe it into the standard input of cvlc or mpv. However cvlc doesn’t really work. It gives all sorts of decoder errors. mpv does work. We can run it as

nc -6 -ulp 8090 | mpv -

We can monitor the traffic received on tun0 with Wireshark.

The best way to check that the streaming data is actually going through the DVB-S2 link is simply to stop Longmynd. This doesn’t touch the tap0 or tun0 interfaces in any way but interrupts the reception. If the stream drops out, that’s an indication that data was being routed as intended.

The Wireshark IO graph shows that the stream traffic is between 200 and 300 kbps. This means that we’re using around 30% of the link capacity.

I have observed occasional dropouts in the audio. These happen together with an error printed by mpv.

[ffmpeg/demuxer] mpegts: PES packet size mismatch
[ffmpeg/demuxer] mpegts: Packet corrupt (stream = 0, dts = 111601322).
A: 00:07:10 / 00:07:11 (100%) Cache: 0.9s/38KB
[ffmpeg/audio] mp3float: Header missing
Error decoding audio.

I haven’t investigated these any further.

Update 2022-11-13: I have discovered and fixed a bug in the defragmentation of Longmynd UDP packets done by dvb-gse. After fixing this, there are no more dropouts or packet loss when streaming.

I have also tried sending 3 simultaneous streams using the same file and 3 different UDP ports (with 3 independent cvlc processes). That gets the link occupation to ~800 kbps and works well. Adding a 4th stream breaks, since we exceed the link capacity.

Finally, I have tried streaming some videos with ffmpeg. A video which is already in a TS of an appropriate bitrate can be streamed with

ffmpeg -re -i video.ts -vcodec copy -acodec copy -f mpegts \
    udp://\[fe80::4f7f:8083:683:69c6%tap0\]:8090

Jumbo frames

GSE can fragment large packets in several BBFRAMEs if necessary, so we can test Jumbo frames if we want. We need to increase the MTU on the tap0 and tun0 interfaces by doing

ip link set tap0 mtu 9000
ip link set tun0 mtu 9000

Then we can send large ping packets with

ping -s 8500 -n 44.0.0.2
ping -s 8500 fe80::1%tap0

and check that they are correctly received in Wireshark (note the length of the packets in the screenshot below).

Appendix: technical details

STV0910 in GSE mode

At first it was unclear whether the MiniTiouner could be used to receive GSE, but Brian G4EWJ noticed that the data sheet for the STV0910 demodulator stated that it supported GSE. He did some tests and found that when the MiniTiouner receives a generic continuous stream, then without any modifications Longmynd outputs BBFRAMEs, including the BBHEADER, to its UDP data output. The BBFRAMEs are fragmented into several UDP packets.

I haven’t checked all the details, but there isn’t much data manipulation between the data output of the STV0910 and Longmynd’s UDP output. The data interface of the STV0910 is connected to the FT2232H, which works in asynchronous FIFO mode to transfer the data output by the STV0910 as USB bulk transfers. This FIFO interface only uses a write strobe line WR# (pin BC3 in the the MiniTiouner schematic) and 8 parallel data lines. On each falling edge of the WR# line, one byte of data is written to the FIFO. The FIFO is then read by USB bulk transfers. The WR# line is driven through some combinational logic by the TS2CLK, TS2VALID and TS2ERR lines of the STV0910.

The STV0910 also has a TS2SYNC line that is pulsed when the start of a TS packet is output (when a TS stream is being received, the output of the STV0910 data interface consists of TS packets rather than BBFRAMEs). However, this TS2SYNC line isn’t used as part of the FIFO interface. There aren’t any kind of frame delimiter markers in the FIFO.

Longmynd basically acts as a pipe that sends everything that it receives from the FT2232H FIFO interface by USB to UDP packets. Longmynd performs bulk transfers of at most 20*512 bytes, and then sends out the data of each bulk transfer as UDP packets with payloads of 510 bytes or less. While doing this, the first 2 bytes in each 512 byte segment of the bulk transfer data is skipped, since it is a header inserted by the FT2232H.

It seems that when the STV0910 receives a generic continuous stream, its output consists of BBFRAMEs, including the BBHEADER. Only the BBHEADER and data field are output. The padding is dropped. For BBFRAMEs in which the data field is empty (data field length equal to zero), which happens whenever no packets are sent in the GSE stream, no output is produced.

In practice, even though there is no frame delimitation in the FIFO interface, it seems that each of these BBFRAMEs is sent out by the FT2232H in a single bulk transfer. The figure below shows the USB transfers for the FIFO (endpoint 0x83) in a test where first a few small ping packets were sent, then a few ping packets with 2000 bytes of payload, and finally a few ping packets with 8500 bytes of payload.

The contents of each bulk transfer correspond to the FTDI USB protocol, which segments each transfer into frames of 512 bytes (the last frame can be shorter), where the first 2 bytes are status, and the remaining 510 bytes are data.

The BBFRAMEs corresponding to the short pings are small, and fit in a single 512 byte segment. The BBFRAMEs for the 2000 byte pings occupy 5 segments. The 8000 byte pings are fragmented (at the GSE level) into 3 BBFRAMEs. The first two are fully occupied. Each is sent out as a single bulk transfer. Note that the 4026 byte size of the large transfers corresponds to the BBFRAME size for rate 1/2 coding with normal FECFRAMEs. Longmynd simply takes the 510 bytes of data in each of the 512 byte segments and sends them out as a UDP packet.

Since there is no explicit frame delimitation in the FIFO, I believe that the fact that BBFRAMEs are nicely aligned with bulk transfers is just a lucky consequence of timing. The STV0910 outputs data (either BBFRAMEs, when receiving a generic continuous stream, or TS packets, when receiving a TS stream) at ~17 Mbit/s. This means that with a DVB-S2 symbol rate of a few Msym/s or less, there is a relatively large gap between BBFRAMEs (or TS packets) on the data interface that goes to the FIFO. If the PC is reading from the USB fast enough, the FIFO always empties before the next BBFRAME comes in, so it never happens that parts of two different BBFRAMEs get mixed in the same bulk transfer.

Therefore, we see that in practice, at the UDP output of Longmynd there are packets of 510 bytes or shorter. Each of these packets either corresponds to the start of a BBFRAME, or a subsequent fragment, but the starts of BBFRAMEs (which contain the BBHEADER) are always aligned with the start of the payload of a UDP packet.

This fact is exploited by dvb-gse, which performs defragmentation of the UDP packets in the following way to obtain full BBFRAMEs. A UDP packet is received, and the start of its data is attempted to be interpreted as a BBHEADER, checking the CRC-8 and other fields (such as whether it is actually a generic continuous stream). If all the checks pass, we assume that this packet corresponds to the start of a BBFRAME, read the data field length, and keep receiving UDP packets until we have all the data for the packet. Then we receive the next UDP packet and check again if it looks like a valid BBHEADER as before. Whenever the header check is not successful, we discard the UDP packet and receive the next, running the BBHEADER checks again, and hoping that at some point we receive a UDP packet that has the start of a BBFRAME.

In higher symbol rates or situations where the PC is heavily loaded, I could imagine parts of different BBFRAMEs being put together in the same bulk transfer. In this case, the BBHEADERs of some BBFRAMEs will no longer be at the start of the payload of a UDP packet, and the defragmentation algorithm of dvb-gse will fail and drop those BBFRAMEs.

The defragmentation algorithm could be improved to handle these situations, but in that case synchronizing up (which needs to be done whenever some UDP packets have been lost or the FT2232 FIFO has overflowed) would be rather expensive, as all the possible offsets need to be tested to try to find a valid CRC-8 (there is no explicit marker at the beginning of BBFRAMEs, in contrary to what happens with TS packets).

Brian also found that by setting bit 5 in the P2_TSINSDELH register of the STV0910, the BBHEADER is omitted from the output. However, I don’t think this is so useful for this use case, because trying to synchronize to the GSE headers in the data field is even harder than trying to synchronize to the BBHEADER. There is no CRC and the length and number of fields in a GSE header is variable.

MAC addresses

GSE supports including MAC addresses (called “labels” in the GSE terminology) in the GSE header of each PDU. These can be used at the receiver to drop GSE packets that aren’t addressed to that particular receiver. Labels can be 6 bytes long (the same size as Ethernet MACs), 3 bytes long, or can be omitted (a situation which is referred to as broadcast GSE packets, since no receivers will drop them).

gr-dvbgse includes 6-byte labels in the GSE packets that it generates. This is the reason why it uses a TAP device (the difference between a TAP device and a TUN device is that a TAP device mimics an Ethernet device, while a TUN device works at the IP level). gr-dvbgse takes the labels for the GSE packets from the destination MAC of the Ethernet frame sent to the TAP device.

However, for simplicity we have disabled ARP in the TAP device, since ARP can’t possibly work in a one-way link (disabling ARP is indicated by the README of gr-dvbgse). This messes up with the way that Linux sets the destination MAC of the Ethernet frames in the TAP device. It seems that unicast packets (both IPv4 and IPv6) use the MAC of the TAP device as destination MAC. Additionally, broadcast IPv4 packets (i.e., those sent to 40.0.0.255) also get the TAP device MAC instead of the broadcast MAC ff:ff:ff:ff:ff:ff as they should. On the contrary, link-local multicast IPv6 packets seem to work well and get the correct multicast MAC. For instance, a packet sent to ff02::1%tap0 gets the destination MAC 33:33:00:00:00:01.

With ARP disabled, it is possible to add manual MAC assignments for IPs as follows:

ip neigh add 44.0.0.2 lladdr 02:00:48:55:4c:4a dev tap0
ip neigh add fe80::1 lladdr 02:00:48:55:4c:4a dev tap0

This will make the Ethernet frames have the expected destination MAC and GSE label.

Currently dvb-gse doesn’t perform any kind of filtering by label. When a GSE packet containing an IP packet is received, in addition to the packet we have the following data, which comes from the GSE headers: the destination label, which is potentially an Ethernet MAC if 6-byte lables are used, and the protocol type, which is an Ethertype (0x0800 for IPv4 and 0x86dd for IPv6). This means that we have almost all the data to generate an Ethernet frame, but not quite. We’re missing the source MAC address.

It is perfectly possible to send Ethernet frames instead of IP packets as the PDU carried by GSE. In this case, the Ethernet frame carries the destination and source MACs and the Ethertype, and the GSE headers repeat the destination MAC as label and the Ethertype as protocol type. This is arguably a waste, because we are sending out 14 additional bytes per packet which are mostly useless.

For this reason, I have decided to use a TUN device for the output of dvb-gse. By doing this, the packets that the kernel network stack receives are IPv4 or IPv6 packets rather than Ethernet frames. The label and protocol types carried in the GSE headers are discarded (the TUN device knows whether the packet we send to it is IPv4 or IPv6 because of the version field in the IP header). With this approach we are unable to use other protocols different from IPv4 and IPv6, even though these could be carried in Ethernet frames and GSE packets by using the appropriate Ethertype / protocol type.

An alternative would be to use a TAP device for the output of dvb-gse and reconstruct Ethernet frames by using a made up source MAC address. Maybe I’ll implement an option to do this in the future.

In the context of using GSE for IP traffic on the QO-100 transponder, I haven’t decided yet whether MAC addresses / GSE labels are potentially useful or just something that gets in the way. For simplicity, we could mostly ignore them (which is what dvb-gse does right now), or even get rid of them by modifying gr-dvbgse to transmit broadcast GSE packets with no labels. It is possible that with some clever management they can provide some practical benefits. Here the idea that I describe in my paper IPv6 for Amateur Radio of encoding amateur radio callsigns in MAC addresses and then using those to derive SLAAC-like IPv6 addresses could be useful.