Reversion in AX.25 Decoding Performance with sensitivity improvements.
Short Version:
- Current gr-satnogs is not reliably decoding AX.25 packets from satellites which transmit intermittently. This appears to be a reversion related to the sensitivity improvement commit ( 118a7ed9 ). Even with observations that clearly have good signals present, the latest gr-satnogs does not reliably decode their packets.
- For satellites that transmit continuously (e.g. FalconSat-3), decoding performance appears to have decreased, though this has only been tested with one observation.
Long Version:
After looking a bit more into why stations running the absolute latest gr-satnogs are showing a lot of false-positive decodes, I decided to try and compare results between pre-change and post-change gr-satnogs using a few example observations.
For examples of some observations with false-positive decodes:
- https://network.satnogs.org/observations/3170476/ (Just noise in this observation)
- https://network.satnogs.org/observations/3175929/ (Just noise in this observation)
- https://network.satnogs.org/observations/3173635/ (Possibly one partially-valid frame, but the rest are all bad)
A good example of a concurrent observation of the same satellite with old and new gr-satnogs versions is here:
- Old: https://network.satnogs.org/observations/3175354/ - 15 valid packets (AX.25 DST address of CQ)
- New: https://network.satnogs.org/observations/3175389/ - 15 gibberish packets
To try and compare the decoder pre/post changes, I set up a test flowgraph based on the fsk demod flowgraph, but using a IQ source. The IQ source is read using a file source (short), then converted to complex using a IShort to Complex type converter. I also added a debug message sink, to output the packets to stdout in JSON format.
These observations used as test IQ sources were:
- SPOC: https://network.satnogs.org/observations/3163929/
- FalconSat-3: https://network.satnogs.org/observations/3175494/
- MCUBED: https://network.satnogs.org/observations/3171035/ Note that the stations above are running older versions of gr-satnogs, but were saving IQ data.
I also wrote a quick test script to print out the src/dst callsign from the AX.25 frame, as a means of checking if the packet was likely valid.
The flowgraph, the log files, and the script are all available here: http://rfhead.net/sats/20201121_ax25_decoder_testing.tar The IQ files are available from: http://satnogs-iq.s3.amazonaws.com/observatio_id.raw
I then ran the flowgraph using the source IQ files above, for the following gr-satnogs commits:
For the first test (SPOC) I also ran it using the commit just after the improvements ( 118a7ed9 ) to verify that it was the one causing the breakage.
Further information on Tests
Flowgraph: fsk_testing.grc (results in satnogs_fsk.py)
The fsk_testing flowgraph includes elements of the fsk demod flowgraph, using signals from the IQ file instead of a soapy source. This flowgraph was run over the source IQ file using a selection of commits from the gr-satnogs history.
When changing gr-satnogs commits, gr-satnogs was completely uninstalled (sudo make uninstall), and then reinstalled (sudo make install, sudo ldconfig) before re-running the flowgraph. Apart from changing the source IQ file, the flowgraph was not changed between tests.
AX.25 Header Decoder
read_json_dump.py written to decode the json outputs, and print out the AX.25 SRC and DST callsigns as a means of quickly checking if the packet is valid.
First Test - 9600 GMSK from SPOC
IQ Source File: Obs 3163929 http://satnogs-iq.s3.amazonaws.com/3163929.raw
Pre-sensitivity improvement results
- Commit: d0b58381
- Results: 7 packets, all valid callsigns.
- Log: pre_sensitivity_improvements.txt
Example Callsigns:
DST: b'COSMO ' SRC: b'MOCI '
DST: b'COSMO ' SRC: b'MOCI '
Post-sensitivity improvements
- Commit: 118a7ed9
- Results: 9 packets passing CRC, no valid callsigns.
- Log: post_sensitivity_changes.txt
Latest Commit
- Commit: a08ea46e
- Results: 7 packets passing CRC, no valid callsigns.
- Log: latest.txt
Example invalid callsigns:
DST: b'Z8;\nJ\x1f' SRC: b'Z+I-Ok'
DST: b'\x0b/\x07c6X' SRC: b'i\x01U#\x1aR'
DST: b'd:oA-\x04' SRC: b'M\x05JY,S'
Second test - FalconSat-3 recording
- Observation ID: 3175494 http://satnogs-iq.s3.amazonaws.com/3175494.raw
Pre-Sensitivity Improvements
- Log: falconsat_pre.txt
- Results: 216 packets, all valid.
Example callsigns:
DST: b'PBLIST' SRC: b'PFS3 '
DST: b'TIME ' SRC: b'PFS3 '
DST: b'LSTAT ' SRC: b'PFS3 '
Latest Commit
- Log: falconsat_latest.txt
- Results: 174 packets, all valid.
Comments
- Interestingly no garbage packets in this test. This might be because FalconSat-3 transmits continuously?
Test 3 - MCUBED
- Observation ID: 3171035 http://satnogs-iq.s3.amazonaws.com/3171035.raw
Pre-Sensitivity Improvements
- Log: mcubed_pre.txt
- Results: 11 packets, all with valid src/dst callsigns.
Example callsigns:
DST: b'CQ ' SRC: b'NOCALL'
DST: b'CQ ' SRC: b'NOCALL'
Latest Commit
- Log: mcubed_latest.txt
- Results: 10 packets, no valid callsigns.
Example callsigns:
DST: b'PsF9;X' SRC: b'Nd8M3`'
DST: b'I^y8F5' SRC: b'#># YT'
DST: b"\x08>'5^\x04" SRC: b'T\x13,\x07SD'