Some fellow Spanish Amateur Operators were talking about the use of the Opera mode as a weak signal mode for the VHF and higher bands. I have little experience with this mode, but I asked them what is the advantage of this mode and how it compares in sensitivity with the JT modes available in WSJT-X. I haven’t found many serious tests of what is the sensitivity of Opera over AWGN, so I’ve done some tests using GNU Radio to generate signals with a known SNR. Here I’ll talk about how to use GNU Radio for this purpose and the results I’ve obtained with Opera. Probably the most interesting part of the post is how to use GNU Radio, because it turns out that Opera is much less sensitive than comparable JT modes.
Opera is a mode which was originally designed as a QRSS mode for the LF and MF bands. QRSS means that the transmission periods are very long, so a huge processing gain can be obtained by using some form of averaging. For instance, one can use really long FFT’s, so the noise power gets spread over many bins, while all the signal power is concentrated in a single bin. Of course, this is a gross simplification and there are many ways to do this sort of processing, but the key idea is that longer transmissions help average noise out. As a rule of thumb, the sensitivity can be increased by 3dB each time that the transmission time is doubled. Therefore, ridiculously high sensitivities can be obtained by using very long transmissions, perhaps spanning several hours. Pieter-Tjer de Boer PA3FWM has some very interesting notes about how coherent BPSK can be used on the VLF band for a very slow QRSS mode which performs near the Shannon limit. His notes also include a small summary of the performance of other Amateur weak signal modes, including Opera and some JT modes.
Ultimately there is a limit to how sensitive a QRSS mode can get by using longer and longer transmissions: frequency instability, either in the form of frequency instabilities in the transmitter and receiver or propagation effects such as Doppler spread. Indeed, using the reasoning above, we see that we only obtain gains as long as we are able to concentrate all the signal in a single FFT bin. If the bins are narrow compared with the frequency stability of the system, then the signal spreads over several FFT bins and no gain is obtained. Of course, frequency stability is harder to achieve as we go up in frequency. For this reason, QRSS modes are more suitable for the MF and lower bands, and normally not very useful for VHF and higher.
Opera uses OOK modulation and some form of FEC, presumably using Walsh matrices. That’s as far as I can read, because the only software that implements the Opera mode is closed-source and documentation with the complete technical specifications of the mode is not available (or at least I haven’t found it). In my opinion, the possibility to study how the different digital modes work is extremely important, since self-training and experimentation are some of the basic pillars of Amateur Radio. This is only possible through complete technical specifications and open-source implementations. Not publishing specifications or releasing closed-source modems for the digital modes is only detrimental to the Amateur Radio community. I fail to see a valid reason why an Amateur operator designing a new digital mode for Amateur Radio use would decide to do this. In contrast, the JT modes have very good technical specifications described in the documentation, and the reference implementations, WSJT-X and company, are open-source software under the GPL licence.
Another problem with closed-source implementations for Amateur radio is the integrity of the contacts. Opera implements a “Deep Search” feature in the 2200m and 630m bands to improve its sensitivity. It seems that what this functionality does is to exchange information in real time over the Internet to get the parts of the message that couldn’t be copied over the air. Clearly this is deceptive at least, probably cheating, and it isn’t allowed in formal definitions of a valid contact for Amateur radio. Since the implementation is closed-source, no one can audit it to check exactly how this Deep Search works and what information is copied over the air precisely. Compare this with the Deep Search functionality in WSJT, which has seen much criticism. This uses a database of active stations, but it doesn’t send information over the Internet in real time. The usual complaint is that the decoder uses the database to fill in the information that couldn’t be copied over the air. This is not as bad as sending the missing information in real time over the Internet. Also, the technical details of the Deep Search in WSJT have been publicly described several times and the source code is there for anyone to audit.
Returning to the technical characteristics of Opera which I do know, OOK (or on-off-keying) modulation means that the mode works by toggling on and off a “continuous wave” for specified periods of time to transmit the data. This is the same method that Morse code (also called CW) uses. In fact, Opera sounds quite Morse-like, as it seems to be based on long and short tones as well. There are several Opera sub-modes. The main difference between the sub-modes is how long it takes to transmit a message. Slower modes are more sensitive, according to the discussion about QRSS above, but they need higher frequency stability, so they are only useful on the lower frequency bands. The slowest modes are really slow, so of course there are also some practicality considerations which depend on the intended application. The name of the sub-modes refers to how long it takes to transmit a message. They are as follows: Op05 (30s), which is recommended for 2m through 23cm; Op1 (1min), for 15m through 4m; Op2 (2min), for 80m through 17m; Op4 (4min), for 160m; Op8 (8min), for 630m; Op16 (16min), for 2200m; Op64 or Op65 (64 or 65min), for 74.5kHz; Op2H (2 hours), for 28.5kHz; and Op4H (4 hours), for 9kHz.
I don’t know how much use has been made of the slowest modes below 2200m. I refer again to the notes by PA3FWM about coherent BPSK, as this mode seems to be the real winner on VLF. According to his data, it would take 78 minutes to transmit the amount of net data contained in an Opera message using his proposed coherent BPSK mode. This makes his BPSK mode slightly slower than Op64. However, it can copy down to a SNR of -57dB in 2.5kHz bandwidth, while Op64 needs -44dB SNR according to the Opera Yahoo group.
When comparing Opera with JT modes, it is important to note that the amount of information transmitted by these modes is quite different. Opera transmits only a callsign. This information can be encoded in a little less than 28 bits. Most JT modes transmit two callsigns and either a grid square or a signal report. Indeed, they transmit 72 bits of information. Therefore, they transmit 2.57 times the amount of information as Opera. Consequently, it is fair in this respect to compare Op05 with JT65, JT9 or QRA64 because Op05 can transmit less information in one minute (transmitting two messages) than a single message in these JT modes (which use one minute periods, and indeed the messages take less than one minute to transmit). An exception is WSPR, which transmits only a callsign, a grid square and a power report, for a total of 50 bits, or 1.79 times an Opera message. The transmit period of WSPR is 2 minutes.
In the Opera Yahoo group there is a list with the sensitivities of the different Opera sub-modes. Here and in other sources there is the statement that since Opera is a 50% duty cycle mode, you should subtract 3dB (or even 6dB) to the decoding threshold SNR of Opera when comparing with 100% duty cycle modes such as the JT modes. This claim makes no sense. To be clear, one has to be precise in the definition of the “signal” part in “signal-to-noise” ratio. Since Opera is an OOK mode, it is more natural to define the “signal” as the power of the tone when a tone is present. Of course, if you take the average power over a whole transmission period, you will get approximately 3dB less signal, since the duty cycle of Opera is roughly 50%. In my opinion, defining the “signal” as the power of each tone is what makes more sense, because it is the method that has always been used to measure CW: one just takes the measure on the S-meter when there is a tone; averaging according to the duty cycle of Morse code is never done.
Another reason to justify this choice is that amplifiers are rated for CW key-down power or SSB PEP power. When you want to compare Opera with JT modes, it’s easier to define the “signal” so that both modes produce the same signal strength with your amplifier, namely its rated CW key-down power (unless due to inadequate cooling you have to derate its power for JT modes and not for Opera according to the higher duty cycle of JT modes). It would be cumbersome to need to remember that your Opera signal strength is half what you get with JT modes (or a continuous carrier, for that matter) just because Opera is silent half of the time.
Another more fundamental reason is that OOK modes work by leaving gaps in the transmission, so of course that in a fixed interval of time you cannot transmit the same amount of energy using OOK than by using a 100% duty cycle mode such as FSK. This is inherent to OOK and a designer choosing to use OOK must know this and live with it. Many of the JT modes spend 50% of the power in synchronization (instead of actually transmitting data) and I haven’t seen any claims that the SNR should be compensated for this.
It would be a different story if one talks about some mode such as BPSK31. This mode has a peak-to-average-power ratio (PAPR) greater than one, because the symbols are shaped to reduce bandwidth. For an idle signal, which changes constantly between both symbols, the PAPR is 2. For a signal with data, the PAPR is a slightly smaller than 2 (but still greater than 1), since the symbol doesn’t change at all transitions. It is common practice to report average power instead of peak-envelope power for BPSK31 or any other signal of such characteristics. Everyone who is active in these modes understands that a 100W PEP amplifier can only produce at most 50W of BPSK31 before noticeable distortion is produced, and typically less than 50W, depending on the linearity of such amplifier. Indeed, for complex OFDM modes such DVB-S, most amplifiers can be used only at a small fraction of their peak output power to prevent huge levels of IMD. It is the responsibility of the user to know how much power he can get from his amplifier while maintaining a clean signal, since this will vary a lot between different amplifiers and modulations. For these modes, the “signal” in SNR is always taken as the average power.
Now let’s see to how to use GNU Radio to generate test files with known SNR. The first step is to record a clean signal with no noise. For this, the transmit signal can be recorded into a WAV file directly. I’m using the Opera software (v1.5.8) inside a virtual machine, because I usually run Linux. I route the audio output of Opera into Audacity, which is running outside the virtual machine, and use Audacity to record the signal at 8kHz sampling rate. This sampling rate is adequate and more or less standard for digital modes that work inside an SSB transceiver’s 2.7kHz passband. In the recording, I leave some seconds of silence before the start of the signal and after the end of the signal.
Now it’s time to measure signal strength with GNU Radio. This is done using the flowgraph below. The instantaneous power of the signal is the square of the amplitude (or rather, it is proportional to the square of the amplitude, but since we are dealing with power ratios we can suppose that the proportionality constant is 1). The instantaneous power is averaged over a period of 10ms. This is a good choice for most signals, since it will average out audio frequencies but preserve features which happen at lower frequencies, such as keying in the case of OOK modes. Using this method, we see that the power of the signal is around 3.45e-5.
The next step is to measure noise power. There are several ways to do this, and one important thing is that we are interested in noise power in a 2.5kHz bandwidth, since that is the standard used when measuring Opera and JT modes. One way to do this is to low-pass filter the noise to 2.5kHz and then measure the noise power. Another possibility is to measure the noise power over the whole bandwidth and then calculate the equivalent noise power for 2.5kHz bandwidth. We are going to generate our test files with noise low-pass filtered to 2.5kHz, so we use the first method.
We measure the noise power using the flowgraph below. It is important to use a long averaging window (in this case 10 seconds) to minimize fluctuations in the average power. We obtain a power of 0.625. Note that we are using a pretty steep low-pass filter.
As we have already mentioned, it is also possible to calculate noise power by doing some math. The Noise Source with an amplitude parameter equal to 1 generates a power of 1 into a 4kHz bandwidth. Therefore, the power in 2.5kHz bandwidth is 2.5/4 = 0.625. This matches the result we have obtained above, so in fact it is not necessary to measure noise power, as it can be calculated directly. However, I like to do it, as it is a good check that I am not making any serious mistakes.
Now we can add the signal and noise at appropriate levels to obtain a test file with a known SNR. Since we are saving the output into a WAV file, it is important that the samples belong to the interval [-1,1] to prevent clipping. To ensure this, we use an amplitude of 0.1 in the Noise Source, so that the probability of a noise sample being outside of [-1,1] is extremely low, and multiply the signal by a suitable factor to achieve the SNR we want. This factor is (0.625*0.01/3.45e-5)**(0.5)*10**(snr/20.0). The flowgraph to generate the test file is below.
Opera doesn’t use fixed time periods such as the JT modes. It can decode signals starting at any moment in time. Hence, it is enough for us to generate a sample file containing many consecutive signals (say 100) with some space in between them to some time to the decoder to process the signal and start with the next one. An easy way to do this is to let the GNU Radio flowgraph run for some time, generating a WAV file which is longer than what we need and then cutting the WAV file to the appropriate length in Audacity. For instance, our recording of an Op05 signal is 40 seconds long, as it contains 5 seconds of silence at the beginning and 5 seconds of silence at the end. Therefore, if we cut our test WAV file at 4000 seconds, we have a test file with 100 Op05 signals. If you use the flowgraph to generate a file with a single transmission and then you run the flowgraph several times, be careful that you don’t generate exactly the same test file every time. You should put a random seed for the Noise Source in this case.
The final step is to play the test file into the Opera decoder and take note of how many successful decodes are done. The cumbersome part of this is that playback and decoding is done in real time. This limits the amount of tests you can do in a reasonable time, especially for the slowest sub-modes. The decoders for the JT modes have the possibility to run on a WAV recording as fast as possible, so it is much easier to do many tests. However, it doesn’t seem that this is possible with the current Opera software, because it only supports audio input.
I have tested the modes Op05, which is useful for VHF and up; Op2, for 80m though 17m; and Op4, for 160m. For each of these modes I’ve prepared a test files at different SNRs and with 100 signals in each file. The results are as follows:
- Op05: -20dB, 8 decodes; -19dB, 22 decodes; -18dB, 67 decodes
- Op2: -26dB, 17 decodes; -25dB, 66 decodes; -24dB, 92 decodes
- Op4: -30dB, 3 decodes; -29dB, 24 decodes; -28dB, 65 decodes
No false decodes where observed.
Therefore, the decoding thresholds for these modes are -19dB for Op05, -25dB for Op2 and -29dB for Op4. In the Opera Yahoo group there is a list with the decoding thresholds of the different modes. My results are 1dB higher than those indicated there.
In a previous post, I looked at the performance of several JT modes. We see that, in general, the JT modes are much more sensitive than Opera, especially Op05. In my opinion, it is pointless to use Op05 in the VHF & up bands. WSPR, JT9A, JT65A and QRA64A outperform it by more than 6dB. WSPR and JT9 may not be appropriate depending on the frequency stability, but JT65A and QRA64A are designed for VHF & up and they will always work well.
In the HF bands, Op2 is a worse performer than WSPR or JT9A. WSPR also has a 2 minute transmit period, but it transmits more information and is 5dB more sensitive than Op2. JT9A uses a 1 minute period and it transmits much more information than Op2. It is also 2dB more sensitive than Op2. Even Op4 doesn’t perform better than WSPR, despite using 4 minutes for the transmit period.