In my talk at GRCon19 last week I presented the roadmap I have planned for gr-satellites for the next months and some longer term developments. The relevant slide can be seen below.
Here I will describe the roadmap in more detail, including how certain things will be done (or how to find your way among the different releases and branches in the Github repository), in order to get feedback from the community.
GNU Radio 3.8
The most immediate item on the roadmap was the port to GNU Radio 3.8 and Python 3. Following Mike Piscopo‘s “Porting OOT modules to GNU Radio 3.8” developer breakout session in GRCon, a first version of gr-satellites supporting GNU Radio 3.8 was made available last Friday. Special thanks to Maitland Bottoms for helping me get GNU Radio 3.8 running in a Debian testing chroot inside my Gentoo machine.
This version is now available on the branch maint-3.8. Most of the decoders have been tested with the satellite-recordings sample files and produce valid decodes. There are still some decoders not tested on GNU Radio 3.8, such as those that don’t have a sample recording or those who need an external GNU Radio OOT module (since these modules do not work with GNU Radio 3.8 yet).
The maint-3.8 branch will only work with GNU Radio 3.8. The branch maint-3.7 contains the version that works with GNU Radio 3.7.
Since it is too much effort for me to maintain both branches, the development will happen on branch maint-3.8. Support for newer satellites will be added in this branch. The first official release of gr-satellites supporting GNU Radio 3.8 will be done on this branch with version number 2.0.0.
Therefore, in principle no more updated releases will be done for GNU Radio 3.7. If backporting some of the changes to maint-3.7 is easy or some satellite is especially demanded, some of the changes might be added to maint-3.7 also, possibly triggering official releases with version number 1.
Thus, the version numbering of gr-satellites according to the version of GNU Radio is simple:
- GNU Radio 3.8: versions 2.*.*
- GNU Radio 3.7: versions 1.*.*
For the time being, the master branch will track maint-3.7, since people are used to building from master and I think I would get complaints of gr-satellites no longer building if I change this now. When GNU Radio 3.8 becomes more popular (i.e., when it gets shipped with major distributions), the master branch will be switched to maint-3.8.
ESA Summer of Code in Space
Athanasios Theocharis has been working with me as part of the ESA Summer of Code in Space to implement new blocks in gr-satellites supporting the CCSDS protocols. You can see the poster that Athanasios presented in GRCon here. This work is based on version 1.5.0 and can be seen in Athanasios’ Github fork.
The ESA SOCIS program will end in September. After this, I will merge Athanasios’ changes into maint-3.7, generating release 1.8.0. This will be perhaps the last official release supporting GNU Radio 3.7. The changes will also be ported to maint-3.8 and release 2.0.0, the first supporting GNU Radio 3.8, will be done.
Refactor
The largest announcement in my talk was that I intend to refactor gr-satellites, doing large changes to the way it is used. This is the part where I am specially interested in getting feedback, since not all design choices have been made yet and I want to know if the changes I plan are useful or not.
I have identified three main use cases for gr-satellites:
- Standalone decoder. This is how many people are using gr-satellites now. The decoder is used as a standalone application getting samples from other software, SDR hardware, or file, and showing decoding information on the screen and output files.
- Building blocks for other GNU Radio decoders. The blocks of gr-satellites are used to create custom decoders using GNU Radio companion flowgraphs. This is how gr-satellites is used by some people who build decoders for their own satellites using some of the blocks from gr-satellites.
- Plugin. This use case is not supported now. The functionality is as in the standalone application, but the decoder is integrated as a plugin in another software such as SatNOGS or GQRX. Therefore, the user interface and API are different.
To support each of the use cases, I have decided the following approaches:
- Standalone decoder. gr-satellites will provide a standalone Python program designed to run from the command line. New satellites will be specified using YAML files or Python code, rather than with flowgraphs. This makes it easier to add new satellites or change common components in several satellites. Command line arguments will be used to select the satellites and input and output options.
- Building blocks for other GNU Radio decoders. The functionality of gr-satellites will be organized in hierarchical flowgraphs or blocks called components (see below), and all the basic building blocks will be available in GNU Radio companion. Example flowgraphs showing how to use some of the main components will be included.
- The approach will be as in the standalone decoder regarding the YAML files or Python code. Still I have not decided anything about the plugin interface, since it is very dependent on what software will use the plugin, but the idea is to reuse as much as possible the standalone decoder code. Hopefully the idea of components will make gr-satellites easier to interface with other tools.
Therefore, the main change with the refactor is that each satellite will no longer have its own flowgraph. Even though I know that some users like to modify the flowgraphs to their liking, the idea of having a flowgraph per satellite has become unmaintainable. With more than 80 different flowgraphs, it is very costly to change some parameter in all the flowgraphs where it is used. Thus, a different approach needs to be followed to support the large number of satellites that we have now. Only example flowgraphs showing how to use the basic components will be included in gr-satellites.
gr-satellites will be organized around the idea of components. These are hierarchical flowgraphs or blocks that perform one of the main functions of the decoding chain:
- Demodulators. They transform samples into soft symbols. No packet boundary detection is made at this stage. For example, FSK demodulator or BPSK demodulator.
- Deframers. They transform soft symbols into frames, detecting packet boundaries and performing error correction and checking as needed. For example, AX.25 demodulator or AX100 demodulator.
- Transports. They are used when the frames carry an upper layer protocol, such as Space Data Link frames carrying Space Packets. They obtain the upper layer packets or data from the lower layer frames.
- Data sinks. They consume data generated by deframers or transports. Examples of this are telemetry parser, image decoders, KISS output to file, and telemetry submitters.
The refactor of gr-satellites aims to provide a solution that is able to decode any of the supported satellites by chaining the appropriate components. The components will be made available as GNU Radio companion blocks so that they can be reused easily in other projects. Different satellites will be specified in terms of what components they need to use as demodulators, deframers and transports, and what data sinks are applicable (the user will be able to select different data sinks to configure the type of output he wants).
The refactor will happen in the next branch of gr-satellites. While it is done (I think it will take several months), support for new satellites will be added in maint-3.8, generating new releases as appropriate. Once the next branch is at a state where it is usable by the wide public, a release with version 3.0.0 will be generated on the next branch. Eventually, the master branch will be moved to next and the development of gr-satellites will continue with versions 3.*.*, so that no additional versions for 2.*.* or 1.*.* will be released.
Thank you for the detailed advance notice of the future form that gr-satellites will take!
“the main change with the refactor is that each satellite will no longer have its own flowgraph”
… given that every user of gr-satellites will have different RF hardware, a different computer environment, and in many cases will desire a different form of output from any given decoder, does a mechanism exist where we could incorporate your command-line Python programs into the GNU Radio GUI environment?
I for one would be much more intimidated by the prospect of having to become functional at a high level with Python code than I have been historically with modifying or augmenting your great GRC flowgraphs.
Thanks!
Hi Scott,
It isn’t expected that most users will need or want to modify the behaviour of the decoders by using the GNU Radio companion. Indeed supporting different RF hardware and output formats is a challenge, but if you think about any other SDR software (GQRX, direwolf, etc.), they manage just fine without letting the user edit internal settings.
Going away from having a self-contained flowgraph for each satellite was not a difficult decision, since I know that there are several users that like to customize the flowgraphs to their liking. However, as I said, having one flowgraph per satellite has become unmaintainable and something more general needs to be done, which requires going away from GNU Radio companion and using Python. However, I still want to respect the wishes of those users that want to customize things.
First of all, following the idea of components explained above, it will be possible to construct a decoder from scratch using a few of the component blocks (for example, an FSK demodulator, an AX.25 deframer, and a telemetry parser for a typical 9k6 packet radio satellite). Advanced users might want to go this way to get a large degree of customizability.
Second, I will consider if it is feasible to arrange things so that it is possible to get a “huge block” in GNU Radio companion that represents the complete decoder for one satellite (with IQ samples as input and packet PDUs as output, and perhaps additional optional outputs to look at intermediate steps of the decoding). Then users might be able to embed the decoder in a larger GNU Radio flowgraph.
Rather than implementing everything at once, I am going to implement first a small subset of the functionality (i.e., decoders for only a few satellites) so that I can see if my ideas work well in practice and users can test the new system. At this stage it will still be possible to change the design due to user feedback.
That sounds great – thank you!
I’ll look forward to using a modular approach to accommodate whatever input and output is needed for a particular signal source.
If you consider three types of users (end-users, curious tinkerers and insiders) then I think the modular components approach will appeal most to the curious tinkerers; no need to completely understand the underpinnings, but still being able to experiment. Right now I consider gr-satellites more for insiders, so the transition will benefit a sizable group. If the end result can be packaged into a turn-key solution, then the end-users can be served as well, thus creating an even larger user base. Looks like a win-win to me.
With that in mind I hope you can consider the following:
1) when designing those new components it would be beneficial to give thought to where and how they can be checked and measured, similar to test points (tp) in electronic circuits. Where are those test points and what is needed (gr sinks) to measure something tangible?
2) gr-satellites is non-GUI based, but as above, some consideration could be done on where and which GUI elements can be added later on to give the end-user some visible information.
Don’t get me wrong, I am not asking for implementation of the above, only that when you start designing you could think of these points and hopefully make some notes so that people who start using these components have a good toolset to use when e.g. trouble shooting or designing a GUI.