Trying to make the DSLWP-B GMSK decoder more robust

If you’ve being following my latest posts, probably you’ve seen that I’m taking great care to decode as much as possible from the SSDV transmissions by DSLWP-B using the recordings made at the Dwingeloo radiotelescope. Since Dwingeloo sees a very high SNR, the reception should be error free, even without any bit error before Turbo decoding.

However, there are some occasional glitches that corrupt a packet, thus losing an SSDV frame. Some of these glitches have been attributed to a frequency jump in the DSLWP-B transmitter. This jump has to do with the onboard TCXO, which compensates frequency digitally, in discrete steps. When the frequency jump happens, the decoder’s PLL loses lock and this corrupts the packet that is being received (note that a carrier phase slip will render the packet undecodable unless it happens very near the end of the packet).

There are other glitches where the gr-dslwp decoder is at fault. The ones that I’ve identify deal in one way or another with the detection of the ASM (attached sync marker). Here I describe some of these problems and my proposed solutions.

The relevant part of the GNU Radio OQPSK decoder for DSLWP-B can be seen below. There are three custom blocks. The first block, “QT GUI FFT Correlator Hier” correlates against the 64bit ASM that marks the beginning of a packet. It uses an FFT to search in the frequency domain. Essentially, it has a matched filter whose taps are formed by the GMSK-modulated ASM. The signal is first routed through this filter, then a block of samples at the output of the filter is taken, and the FFT of this block is calculated. A peak in the FFT indicates a correlation at the particular frequency corresponding to the bin where the peak has occured and the particular time corresponding to the samples where the block has been taken. This algorithm is very similar to the one I use for my GMSK detector.

When the Correlator Hier block detects a correlation peak, it sends tags with the correlation parameters to the downstream blocks. These tags inform of the estimated frequency and phase, signal amplitude and Eb/N0, and (implicitly) the sample where the ASM starts.

The “Set Gain by Tag CC” block tries to set an amplitude level of one in the input of the OQPSK demodulator. It uses the signal amplitude tags sent by the Correlation Hier block to obtain the estimated signal amplitude and then divides the samples by this value.

The OQPSK demodulator is based on a OQPSK implementation in Octave by David Rowe VK5DGR. It performs bit shaping matched filter, extraction of soft symbols, and optionally carrier tracking with a PLL. Clock recovery (DTLL) is not implemented and it doesn’t seem necessary, since the symbol clock doesn’t drift much during each packet. The demodulator uses the frequency and phase estimates from the Correlator Hier block to set the initial values of the carrier tracking. The PLL is needed, since otherwise the phase quickly drifts out. Anytime a new frequency and phase estimate is received, it overrides whatever the PLL was doing.

The sample where the ASM starts (as detected by the correlator) is used for open loop clock recovery. The phase of the clock is set according to that estimate and the symbol clock proceeds at is nominal frequency of 16 samples per symbol (at 2kHz sample rate).

The main issue that I have noted is that the Correlator Hier is very sensitive. In order to detect very weak packets, its threshold is usually set to -2dB Eb/N0. Integrating over 64 symbols of ASM this gives a correlation SNR of 16dB, which is enough for reliable decoding. However, false detections are also possible, especially at this low threshold.

Indeed, I think that the high SNR seen at Dwingeloo makes false detections during the GMSK data more likely, because of the cross-correlation between the ASM symbols and some of the payload symbols.

For my tests I have used several of the latest Dwingeloo recordings containing SSDV downloads, but for this post I will use the 2018-08-14T08_59_11_436.4MHz recording as a reference.

The image below shows one of the problems I have encountered. An ASM false detection happens near the right side of the image. Note that up to that point the ASMs keep a constant period, since the payloads are always the same length. Then there is a false detection of an ASM. This sends incorrect frequency and phase estimates to the OQPSK demodulator, which loses carrier lock. The amplitude estimate is also too low, so the Set Gain block sets a higher gain to try to compensate. The effect is that the packet affected by this false detection is corrupted and it can’t be decoded. When the next ASM is detected, decoding resumes back to normal.

I am trying different ways to mitigate these problems. A simple idea is to raise the Correlator Hier threshold. Since the SNR seen at Dwingeloo is very high, we don’t need a threshold as low as -2dB Eb/N0. However, this doesn’t work well. Setting a threshold of 5dB Eb/N0 already misses some ASMs even though the Eb/N0 is always above 20dB. I will have to look more closely to see if this can be improved.

Another easy idea is that the decoder should ignore ASM detections that happen while it is already processing a packet payload. For processing single packets, this approach doesn’t make much of a difference. Assume that a false ASM detection happens near a packet. If we use all ASM detections, we will get the packet correctly if and only if the false detection happens before the packet. If we ignore ASM detection while processing the payload, we will get the packet if and only if the false detection happens after the packet. These two situations are equally likely (unless you consider that a false correlation is more likely during GMSK data because of cross-correlation).

However, the situation is different for a continuous stream of packets, as used for SSDV. The approach of using all ASM detections corrupts a packet anytime that a false detection happens inside a packet, so we can get multiple corruptions in the stream. If we ignore ASMs during payloads, the only possible packet that can be lost is the first one, and only when the false detection happens right before it (assuming that in this case a false detection doesn’t happen also before the second packet, which is not very likely).

I have implemented an option in the OQPSK decoder to ignore ASM detections until a given number of symbols after the last (valid) ASM detection. This option is controlled by the field “ASM ignore (symbols)”. Setting this value to zero results in the old behaviour of using all ASMs detections. Setting it to 1818 symbols ignores all ASMs during the payload, while allowing for the very unlikely event of 2 symbols of clock slip (the number of symbols between ASMs is 1820, since the payload is 3576 bits long, the ASM is 64 bits long, and every QPSK symbol carries 2 bits).

In the image below we can see the same situation with this option enabled. The false ASM detection happens in the same place. However, it is ignored and not passed on by the OQPSK decoder. Note that only the harmless payload_start tag remains in the spot of the false detection. The phase hasn’t slipped this time, since the PLL hasn’t been disturbed. However, the false detection is still visible because the “Set Gain” block hasn’t ignored this detection. The packet is now decoded correctly.

It would be good to make the “Set Gain” block ignore ASMs in the same manner. However, I don’t want to implement the same logic in the two blocks. I’m tempted of removing the “Set Gain” block and integrating its functionality inside the OQPSK demodulator block.

The test of this improvement has been performed with the 2018-08-14T08_59_11_436.4MHz recording. Using all ASMs a total of 118 SSDV packets are decoded. With the modification, a total of 121 SSDV packets are decoded, so essentially we’ve solved a few glitches.

For a visual comparison, see the two images below. The first one was decoded without the improvement and the second one with it. Note that there are three “strips” which are missing in the first one but not in the second one. These correspond to the three glitches fixed by the improvement, which result in three more decoded packets. Still, it seems that one packet could not be decoded for some reason.

Although this easy modification seems to work well, I think that there is still much that can be studied and done to try to improve the robustness of the decoder even further. Perhaps there are some algorithms that work better for the ASM detector. Also, the Eb/N0 estimate can be improved. It was designed for low SNRs, and it sometimes give very high (or even negative) values for the high SNR signals received by Dwingeloo.

The modification described in this post can be found in my asm_robustness branch of gr-dslwp. I will now send a pull request of this branch to Wei Mingchuan BG2BHC in case he wants to include it in the upstream decoder. If your compiler is as mine and gives some compile errors with gr-dslwp, try instead my compile_fixes branch, which includes asm_robustness and also fixes these errors.

Update 2018-08-17: The modification has been included in the upstream gr-dslwp master branch, together with the compilation fixes, so I will probably delete the two branches mentioned above soon.