All jobs fail after cb transfer status: 2, canceling...
After the satnogs-client runs into a cb transfer status: 2, canceling..
error,
all further jobs fail with and error satnogsclient.scheduler.tasks - ERROR - Observer job lock acquiring timed out
and the software needs a restart sudo systemctl restart satnogs-client
This problems happens as it seems to be randomly.
If further information is required, please let me know.
System information:
------------[ copy here ]------------
{
"versions": {
"satnogs-client": "1.7",
"satnogs-client-ansible": "202201270927",
"satnogs-flowgraphs": "1.4-1",
"gr-satnogs": "2.3.1.1-1",
"gr-soapy": "2.1.3.1+2+gf2c6097a-1",
"gnuradio": "3.8.2.0-14satnogs2",
"satnogs-config": "0.12+2.ga40a9e9"
},
"state": {
"is-applied": true,
"pending-tags": null
},
"system": {
"date": "2022-02-25T09:02:24.542813+00:00",
"distribution": {
"DESCRIPTION": "Debian GNU/Linux 10 (buster)",
"RELEASE": "10",
"CODENAME": "buster",
"ID": "Debian"
},
"pending-updates": true,
"platform": {
"system": "Linux",
"node": "hostname",
"release": "4.19.0-18-amd64",
"version": "#1 SMP Debian 4.19.208-1 (2021-09-29)",
"machine": "x86_64",
"processor": ""
},
"memory": {
"total": 4016156672,
"available": 3412996096,
"percent": 15.0,
"used": 326832128,
"free": 2920488960,
"active": 530309120,
"inactive": 345632768,
"buffers": 138735616,
"cached": 630099968,
"shared": 29036544,
"slab": 136781824
},
"disk": {
"total": 2080456704,
"used": 16752640,
"free": 1941737472,
"percent": 0.9
}
},
"configuration": {
"experimental": true,
"satnogs_antenna": "RX",
"satnogs_api_token": "[redacted]",
"satnogs_dev_args": "biastee=true",
"satnogs_post_observation_script": "/usr/local/bin/satnogs-post {{ID}} {{FREQ}} {{TLE}} {{TIMESTAMP}} {{BAUD}} {{SCRIPT_NAME}}",
"satnogs_ppm_error": "0",
"satnogs_pre_observation_script": "/usr/local/bin/satnogs-pre {{ID}} {{FREQ}} {{TLE}} {{TIMESTAMP}} {{BAUD}} {{SCRIPT_NAME}}",
"satnogs_rf_gain": "29.7",
"satnogs_rx_samp_rate": "2.048e6",
"satnogs_soapy_rx_device": "driver=rtlsdr,serial=00001220",
"satnogs_station_elev": "5",
"satnogs_station_id": "2176",
"satnogs_station_lat": "[redacted]",
"satnogs_station_lon": "[redacted]",
"snmpd_enabled": true,
"snmpd_rocommunity": "[redacted]",
"udp_dump_host": "127.0.0.1",
"udp_dump_port": "57356"
}
}
------------[ copy end ]-------------
Log information:
Feb 25 07:36:03 hostname satnogs-client[5361]: gr-satellites: Observation: 5533447, Norad: 39433, Name: 0_HUMSAT_D, Script: satnogs_afsk1200_ax25.py
Feb 25 07:36:03 hostname satnogs-client[5361]: gr-satellites: Starting observation 5533447
Feb 25 07:36:03 hostname satnogs-client[5361]: gr-satellites: running at 48000 sps
Feb 25 07:36:04 hostname satnogs-client[5361]: Found Rafael Micro R820T tuner
Feb 25 07:36:05 hostname satnogs-client[5361]: Found Rafael Micro R820T tuner
Feb 25 07:36:05 hostname satnogs-client[5361]: [INFO] Opening Generic RTL2832U OEM :: 00001220...
Feb 25 07:36:05 hostname satnogs-client[5361]: Found Rafael Micro R820T tuner
Feb 25 07:36:06 hostname satnogs-client[5361]: [R82XX] PLL not locked!
Feb 25 07:36:06 hostname satnogs-client[5361]: [INFO] Using format CF32.
Feb 25 07:36:06 hostname satnogs-client[5361]: Allocating 15 zero-copy buffers
Feb 25 07:36:16 hostname satnogs-client[5361]: cb transfer status: 2, canceling...
Feb 25 07:40:29 hostname satnogs-client[5361]: Resetting device...
Feb 25 07:40:30 hostname satnogs-client[5361]: gr-satellites: Observation: 5533447, Norad: 39433, Name: 0_HUMSAT_D, Script: satnogs_afsk1200_ax25.py
Feb 25 07:40:30 hostname satnogs-client[5361]: gr-satellites: Stopping observation 5533447
Feb 25 07:40:30 hostname satnogs-client[5361]: satnogsclient.observer.observer - ERROR - Waterfall data array is empty
Feb 25 07:40:31 hostname satnogs-client[5361]: apscheduler.executors.default - ERROR - Job "spawn_observer (trigger: date[2022-02-25 07:36:03 UTC], next run at: 2022-02-25 07:36:03 UTC)" raised an exception
Feb 25 07:40:31 hostname satnogs-client[5361]: Traceback (most recent call last):
Feb 25 07:40:31 hostname satnogs-client[5361]: File "/var/lib/satnogs/lib/python3.7/site-packages/apscheduler/executors/base.py", line 125, in run_job
Feb 25 07:40:31 hostname satnogs-client[5361]: retval = job.func(*job.args, **job.kwargs)
Feb 25 07:40:31 hostname satnogs-client[5361]: File "/var/lib/satnogs/lib/python3.7/site-packages/satnogsclient/scheduler/tasks.py", line 65, in spawn_observer
Feb 25 07:40:31 hostname satnogs-client[5361]: observer.observe()
Feb 25 07:40:31 hostname satnogs-client[5361]: File "/var/lib/satnogs/lib/python3.7/site-packages/satnogsclient/observer/observer.py", line 261, in observe
Feb 25 07:40:31 hostname satnogs-client[5361]: artifact = Artifacts(waterfall, metadata)
Feb 25 07:40:31 hostname satnogs-client[5361]: File "/var/lib/satnogs/lib/python3.7/site-packages/satnogsclient/artifacts.py", line 30, in __init__
Feb 25 07:40:31 hostname satnogs-client[5361]: self._waterfall_data = waterfall.data
Feb 25 07:40:31 hostname satnogs-client[5361]: AttributeError: 'NoneType' object has no attribute 'data'
Feb 25 08:17:12 hostname satnogs-client[5361]: satnogsclient.scheduler.tasks - ERROR - Observer job lock acquiring timed out.
Feb 25 08:44:42 hostname satnogs-client[5361]: satnogsclient.scheduler.tasks - ERROR - Observer job lock acquiring timed out.
Feb 25 08:50:22 hostname satnogs-client[5361]: satnogsclient.scheduler.tasks - ERROR - Observer job lock acquiring timed out.