README.md 6.77 KB
Newer Older
1
2
3
4
# SatNOGS Decoders

Kaitai Structs, preprocessors and helper scripts for decoding SatNOGS received data.

5
6
## Adding a new decoder

Patrick Dohmen's avatar
Patrick Dohmen committed
7
8
9
10
11
12
13
14
1. Write the kaitai structure and add it to the ksy folder.
   - Naming example:
     * Satellite Name (arbitrary): `CubeBel-1`
     * KS filename: `cubebel1.ksy`
     * KS file header: see [1]
     * KS compiler output: `cubebel1.py`
     * Python capitalized function name: `Cubebel1`
     * The python module name should match the python function naming rules in [PEP8](https://www.python.org/dev/peps/pep-0008/#id40).
15
16
17
18
19
20
21
22
23
24
25
26
   - Add documentation fields in kaitai structure
     * Add "doc-ref", is the source of information about the decoder
     * Add -orig-id, is the name of field which is the same with the source of information, "doc-ref"
     * At the beginning of kaitai struct after the meta data it is necessary to add, the way that the influxdb value it matches to the kaitai field. For example:
     ```yaml
     doc: |
       :field influxdb_field: katai_field

     doc: |
       :field dest_callsign: ax25_frame.ax25_header.dest_callsign_raw.callsign_ror.callsign
    ```

Patrick Dohmen's avatar
Patrick Dohmen committed
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
2. Add the python class name of your decoder to the list in `satnogsdecoders/decoder/__init__.py`

`__init__.py`:
```python
__all__ = [
    [..],
    'Cubebel1',
    [..],
]

[..]
from .cubebel1 import Cubebel1
[..]
```

[1] `cubebel1.ksy`:
```yaml
meta:
  id: cubebel1
```
47

48
49
## Installation in development mode

50
Within the root directory of this repository run `docker-ksc` script to compile KSY to Python code (requires Docker):
51
```
52
$ ./contrib/docker-ksc.sh
53
54
55
56
57
58
59
```
The above command will output the compiled files under `satnogsdecoders/decoder` directory.

Then, install the package from source code directory as usual:
```
pip install -e .
```
60
61

## Helper Scripts
Yuri-M-Dias's avatar
Yuri-M-Dias committed
62
63

Some helper commands will only be available after installing the package.
64

65
66
67
68
69
70
71
### Installation

The helper scripts need a few additional dependencies which can be installed via pip:
```
pip install -r contrib/manage/requirements.txt
```

Fabian P. Schmidt's avatar
Fabian P. Schmidt committed
72
### Fetch telemetry
73

74
To fetch telemetry you will need an API Token. You will have to register yourself on [https://db.satnogs.org](https://db.satnogs.org) in order to get it.
deck's avatar
deck committed
75

76
Once you have your token create a new file `.env` in the current directory based on "./contrib/manage/env-dist" and add your token:
deck's avatar
deck committed
77
78

```
79
SATNOGS_DB_API_TOKEN=1234567890asdfghjkl
deck's avatar
deck committed
80
81
82
```


Fabian P. Schmidt's avatar
Fabian P. Schmidt committed
83
```
84
$ ./contrib/manage/fetch_telemetry.py --help
85
usage: fetch_telemetry.py [-h] [--source SOURCE] [--base_dir BASE_DIR] [--max MAX] norad_id
Fabian P. Schmidt's avatar
Fabian P. Schmidt committed
86

87
Fetch and store all telemetry data from a satnogs-db instance for a given satellite.
Fabian P. Schmidt's avatar
Fabian P. Schmidt committed
88
89
90
91
92
93
94
95

positional arguments:
  norad_id             NORAD ID of the satellite

optional arguments:
  -h, --help           show this help message and exit
  --source SOURCE      satnogs-db Instance: satnogs, satnogs-dev or sputnix
  --base_dir BASE_DIR  Base directory of the telemetry storage
96
  --max MAX            Maximum number of fetched frames. Default: 25
Fabian P. Schmidt's avatar
Fabian P. Schmidt committed
97
```
98
99


Fabian P. Schmidt's avatar
Fabian P. Schmidt committed
100
### Push telemetry
101

Fabian P. Schmidt's avatar
Fabian P. Schmidt committed
102
```
103
$ ./contrib/manage/push_telemetry.py --help
104
usage: push_telemetry.py [-h] [--target TARGET] [--max MAX] telemetry_filename norad_id_import norad_id_export
105

106
Push all telemetry data to the target db instance from a local json dump file for a given satellite.
107
108
109
110
111
112
113
114

positional arguments:
  telemetry_filename  Filname of the telemetry data json dump
  norad_id_import     NORAD ID of the satellite in the data
  norad_id_export     NORAD ID of the satellite in the target db instance

optional arguments:
  -h, --help          show this help message and exit
115
116
  --target TARGET     target satnogs-db Instance: satnogs, satnogs-dev or sputnix
  --max MAX           Maximum number of frames to be submitted.
Fabian P. Schmidt's avatar
Fabian P. Schmidt committed
117
118
```

119

Fabian P. Schmidt's avatar
Fabian P. Schmidt committed
120
121
122
### Export raw frames

```
123
$ ./contrib/manage/export_raw.py --help
Fabian P. Schmidt's avatar
Fabian P. Schmidt committed
124
125
126
127
128
129
130
131
132
133
134
135
136
usage: export_raw.py [-h] norad_id source_file satellite_name

Export raw frames from local telemtry storage json files.

positional arguments:
  norad_id        NORAD ID of the satellite
  source_file     Source telemetry storage file (json)
  satellite_name  Satellite name

optional arguments:
  -h, --help      show this help message and exit
```

Fabian P. Schmidt's avatar
Fabian P. Schmidt committed
137
138
### Decode frame

Yuri-M-Dias's avatar
Yuri-M-Dias committed
139
`decode_frame` will be available as a command line tool after installing the package.
140

Fabian P. Schmidt's avatar
Fabian P. Schmidt committed
141
```
142
$ decode_frame --help
Yuri-M-Dias's avatar
Yuri-M-Dias committed
143
usage: decode_frame [-h] decoder_name raw_frame_file
Fabian P. Schmidt's avatar
Fabian P. Schmidt committed
144

145
146
Decode a raw frame with the selected decoder(generated by Kaitai) and print
its json representation.
Fabian P. Schmidt's avatar
Fabian P. Schmidt committed
147
148

positional arguments:
149
150
  decoder_name    name of the decoder (e.g. siriussat)
  raw_frame_file  Path to the file containing the raw frame
Fabian P. Schmidt's avatar
Fabian P. Schmidt committed
151
152
153
154

optional arguments:
  -h, --help      show this help message and exit
```
155

156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
### Fetch frames from network

```
$ ./contrib/manage/fetch_frames_from_network.py --help
usage: fetch_frames_from_network.py [-h] norad_id start end target_dir

Fetch all frames received in thespecified timeframe from a given satellite in
satnogs-network (prod) and store themto individual raw files.

positional arguments:
  norad_id    NORAD ID of the satellite
  start       Start date, YYYY-mm-dd
  end         End date, YYYY-mm-dd
  target_dir  target directory for the downloaded raw frames

optional arguments:
  -h, --help  show this help message and exit
```

175
176
177
### Example Usage

Transfer frames from db-dev to db-dev (duplicating frames...):
178
```
179
$ ./contrib/manage/fetch_telemetry.py 40967 --source satnogs-dev --max 10 --base_dir ./telemetry/
180
181
182
183
184
https://db-dev.satnogs.org/api/telemetry/?satellite=40967
Fetched 25 frames.
Stored in ./telemetry/satnogs-dev/40967/20180927195606_all_telemetry.json


185
$ ./contrib/manage/push_telemetry.py ./telemetry/satnogs-dev/40967/20180927195606_all_telemetry.json 40967 40967 --target satnogs-dev --max 3
186
187
188
189
0 SR1GEO_DEV01-JO73mi 2018-09-20T21:40:24Z
1 SR1GEO_DEV01-JO73mi 2018-09-20T21:40:19Z
2 SR1GEO_DEV01-JO73mi 2018-09-20T21:40:14Z
Exported 3 frames.
190
```
191

192
193
Decode a frame by Siriussat:
```
194

195
$ decode_frame siriussat contrib/siriussat/packets/data_219992_2018-08-22T13-46-52
196
197
198
199
200
{
    "dest_callsign": "R2ANF ",
    "src_callsign": "RS13S ",
    ...
}
201
```
202
203
204
205
206
207
208
209

Download frames from satnogs-network (prod instance) for Fox-1A (norad id: 40967) received in the specified timeframe:
```
$ mkdir fox1a
$ ./contrib/manage/fetch_frames_from_network.py 40967 2018-10-26T00:00:00 2018-10-26T01:00:00 ./fox1a/
Fetched 45 frames.
```

210
## Existing decoders
211
- AAUSAT4 by OZ3RF & DL4PD
Patrick Dohmen's avatar
Patrick Dohmen committed
212
- ACRUX-1 by DL4PD
deck's avatar
deck committed
213
- AMICALSAT by deckbsd
Patrick Dohmen's avatar
Patrick Dohmen committed
214
- ARMADILLO by DL4PD
215
- ASU PHOENIX by DL4PD & deckbsd
216
217
- AMSAT FOX DUV by DL4PD
- AX.25 frame decoder by DL4PD
218
- CAS-4A & CAS-4B by cshields
Patrick Dohmen's avatar
Patrick Dohmen committed
219
- CHOMPTT by DL4PD
220
- CubeBel-1 by DL4PD
Pierros Papadeas's avatar
Pierros Papadeas committed
221
- Delfi N3xt by pierros
222
- Elfin-A & -B by DL4PD
Patrick Dohmen's avatar
Patrick Dohmen committed
223
- Entrysat by DL4PD
Dillan McDonald's avatar
Dillan McDonald committed
224
- GT-1 by dillan1
Patrick Dohmen's avatar
Patrick Dohmen committed
225
- Lightsail-2 by DL4PD
deck's avatar
deck committed
226
- Ops-sat by deckbsd
Patrick Dohmen's avatar
Patrick Dohmen committed
227
- Painani by DL4PD
deck's avatar
deck committed
228
- Polyitan-1 by deckbsd
229
- QBEE by Ansgar Schmidt
230
231
- Siriussat-1 & -2 by kerel
- skCUBE by borispilka & kerel
232
- Strand-1 by kerel
Dillan McDonald's avatar
Dillan McDonald committed
233
- TARGIT by dillan1
Patrick Dohmen's avatar
Patrick Dohmen committed
234
- TBEX-A/-B by DL4PD
235
- Unisat-6 by cshields
236
237

## License
238
Helper scripts: AGPL-3.0-or-later