Decoding STEREO-A SECCHI images

STEREO-A is a NASA solar observation spacecraft that was launched in 2006 to a heliocentric orbit with a radius of about 1 au. One of the instruments, called SECCHI, takes images of the Sun using different telescopes and coronagraphs to study coronal mass ejections. Near real-time images of this instrument can be seen in this webpage. These images are certainly the most eye-catching data transmitted by STEREO-A in its X-band space weather beacon.

In September 2022, Wei Mingchuan BG2BHC made some recordings of the space weather beacon using a 13 meter antenna in Harbin Institute of Technology. I wrote a blog post showing a GNU Radio decoder and a Jupyter notebook analysing the decoded frames. I managed to figure out that some of the data corresponded to the S/WAVES instrument, which is an electric field sensor with a spectrometer covering 125 kHz to 16 MHz.

During this summer, STEREO-A is very close to Earth for the first time since it was launched. This has enabled amateur observers with small stations to receive and decode the space weather beacon. As part of these activities, Alan Antoine F4LAU and Scott Tilley VE7TIL have figured out how to decode the SECCHI images. Scott has published a summary of this in his blog, and Alan has added a decoding pipeline to SatDump. Using this information, I have now extended my Jupyter notebook to decode the SECCHI images. Eventually I want to turn this into a GNU Radio demo, and the Jupyter notebook serves as reference code.

The main documentation explaining how the SECCHI images are transmitted in the space weather beacon is the STEREO SECCHI/L0 to L0.5 Calibration and Measurement Algorithm Document (this seems to date from 2021, which is relatively recently, given the age of the mission). This document describes pretty much all that is needed to obtain the compressed image files from the telemetry frames, though some comments can serve as a complement and make this document easier to read.

SECCHI images are transmitted in Space Packets in the following APIDs, as Scott has figured out:

  • APID 1136. COR1 coronagraph (though it appears to be unused)
  • APID 1137. COR2 coronagraph.
  • APID 1138. HI1 heliospheric imager.
  • APID 1139. HI2 heliospheric imager.
  • APID 1140. EUVI extreme UV imager.

In addition to this, the other instruments use APID 624 (IMPACT), APID 880 (PLASTIC), and APID 1393 (S/WAVES), while spacecraft telemetry goes in APID 0.

Each image is divided in three blocks: a header, which contains some metadata from which a FITS header can be produced; a block with the compressed image data; and a trailer, which is a copy of the header. The metadata is not essential to obtain and display bitmap images, so I haven’t taken a look at the headers.

These blocks are carried in Space Packets in the corresponding APIDs, and fragmented and aggregated using a system based on a first header pointer, very much like how TM frames carry Space Packets. The structure of the SECCHI Space Packets is shown below. The Primary Header is the Space Packet Primary Header. The first header pointer (first block pointer) field points to the first byte of the first block header (field of 0’s and image count) occurring in the packet payload, if any.

SECCHI Space Packet format (taken from this document)

The block header format seems to support transmitting images in multiple blocks, since it has a 14-bit block number. However, it seems that this isn’t done. The header is transmitted in block number 0 (with block type 1), then all the data is transmitted in block number 1 (with block type 0), and finally the trailer is transmitted in block number 2 (with block type 2).

The documentation describes different compression algorithms for the image data, but it seems that the one that is used normally is ICER. This is a wavelet-based image compression algorithm with some slight similarities to JPEG-2000. As far as I know, an open source implementation of this algorithm is not available, and some people say that there are patents surrounding the algorithm, though perhaps they have already expired. There is this relatively new work-in-progress open source implementation of ICER. I haven’t tried to use it, since the documentation says that it only supports 5-6 bits per pixel, while SECCHI uses 13-14 (and besides this, I’m not sure if this implementation is intended to be interoperable with other ICER implementations).

One of the key findings of Alan was that there is a closed-source ICER implementation in the STEREO website. This is available as an i386 Linux binary only. The assembly code of the decompressor looks relatively simple. Not in the sense that reading it makes obvious how the algorithm works, but in the sense that decompiling it to compile it for other platforms is not a too far fetched idea. These are the only library functions that the binary calls:

$ objdump -T idecomp.linux 

idecomp.linux:     file format elf32-i386

0804855c      DF *UND*	0000006d (GLIBC_2.0)  feof
0804856c      DF *UND*	00000023 (GLIBC_2.0)  fprintf
0804857c      DF *UND*	000001ab (GLIBC_2.0)  malloc
0804858c      DF *UND*	0000003f (GLIBC_2.0)  scanf
0804859c      DF *UND*	0000010b (GLIBC_2.0)  fread
08051160 g    DO .bss	00000004 (GLIBC_2.0)  stderr
080485ac      DF *UND*	000000f8 (GLIBC_2.0)  fseek
080485bc      DF *UND*	00000046 (GLIBC_2.0)  getopt
080485cc      DF *UND*	000000f3 (GLIBC_2.0)  __libc_start_main
080485dc      DF *UND*	00000039 (GLIBC_2.0)  printf
080485ec      DF *UND*	00000180 (GLIBC_2.1)  fclose
080485fc      DF *UND*	000000cc (GLIBC_2.0)  exit
0804860c      DF *UND*	000002e0 (GLIBC_2.0)  calloc
0804861c      DF *UND*	00000034 (GLIBC_2.0)  sscanf
0804862c      DF *UND*	000000b1 (GLIBC_2.0)  free
0804863c      DF *UND*	00000036 (GLIBC_2.1)  fopen
08051164 g    DO .bss	00000004 (GLIBC_2.0)  optarg
0804c5c4 g    DO .rodata	00000004  Base        _IO_stdin_used
0804864c      DF *UND*	00000165 (GLIBC_2.0)  fwrite
00000000  w   D  *UND*	00000000              _Jv_RegisterClasses
00000000  w   D  *UND*	00000000              __gmon_start__
0804865c      DF *UND*	00000030 (GLIBC_2.0)  strcpy

In any case, using this ICER decompressor binary gives a way to obtain the SECCHI images from the STEREO-A telemetry.

The output of the ICER decompressor is a 16-bit grayscale image of square size (the resolution depends on the sensor). The images need to be rotated 90 degrees, and perhaps flipped (depending on the sensor) to obtain the usual north up orientation (though I haven’t seen this documented anywhere).

I have updated the Jupyter notebook to extract the image blocks from the SECCHI APIDs, call the ICER decompressor, and plot the bitmap data. Since the recordings that Wei made are not very long, and the space weather beacon data rate is rather low, there are only a few images.

APID 1137 (COR2) contains the following blocks:

Container(padding=None, image_count=806, block_type=2, block_number=2, block_length=215),
Container(padding=None, image_count=2047, block_type=0, block_number=0, block_length=111),
Container(padding=None, image_count=879, block_type=1, block_number=0, block_length=215),
Container(padding=None, image_count=879, block_type=0, block_number=1, block_length=12139),
Container(padding=None, image_count=879, block_type=2, block_number=2, block_length=215),
Container(padding=None, image_count=2047, block_type=0, block_number=0, block_length=45)

This gives the following image.

APID 1140 (EUVI) contains the following blocks (the data block for imagee 810 was lost because some Space Packets were lost):

Container(padding=None, image_count=810, block_type=1, block_number=0, block_length=363),
Container(padding=None, image_count=810, block_type=2, block_number=2, block_length=363),
Container(padding=None, image_count=2047, block_type=0, block_number=0, block_length=167),
Container(padding=None, image_count=858, block_type=1, block_number=0, block_length=363),
Container(padding=None, image_count=858, block_type=0, block_number=1, block_length=10443),
Container(padding=None, image_count=858, block_type=2, block_number=2, block_length=363),
Container(padding=None, image_count=2047, block_type=0, block_number=0, block_length=155),
Container(padding=None, image_count=901, block_type=1, block_number=0, block_length=363),
Container(padding=None, image_count=901, block_type=0, block_number=1, block_length=10383),
Container(padding=None, image_count=901, block_type=2, block_number=2, block_length=363),
Container(padding=None, image_count=2047, block_type=0, block_number=0, block_length=215)

These give the following two images:

Leave a comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.