All jobs fail after cb transfer status: 2, canceling...

After the satnogs-client runs into a cb transfer status: 2, canceling.. error, all further jobs fail with and error satnogsclient.scheduler.tasks - ERROR - Observer job lock acquiring timed out and the software needs a restart sudo systemctl restart satnogs-client

This problems happens as it seems to be randomly.

If further information is required, please let me know.

System information:

------------[ copy here ]------------
{
    "versions": {
        "satnogs-client": "1.7",
        "satnogs-client-ansible": "202201270927",
        "satnogs-flowgraphs": "1.4-1",
        "gr-satnogs": "2.3.1.1-1",
        "gr-soapy": "2.1.3.1+2+gf2c6097a-1",
        "gnuradio": "3.8.2.0-14satnogs2",
        "satnogs-config": "0.12+2.ga40a9e9"
    },
    "state": {
        "is-applied": true,
        "pending-tags": null
    },
    "system": {
        "date": "2022-02-25T09:02:24.542813+00:00",
        "distribution": {
            "DESCRIPTION": "Debian GNU/Linux 10 (buster)",
            "RELEASE": "10",
            "CODENAME": "buster",
            "ID": "Debian"
        },
        "pending-updates": true,
        "platform": {
            "system": "Linux",
            "node": "hostname",
            "release": "4.19.0-18-amd64",
            "version": "#1 SMP Debian 4.19.208-1 (2021-09-29)",
            "machine": "x86_64",
            "processor": ""
        },
        "memory": {
            "total": 4016156672,
            "available": 3412996096,
            "percent": 15.0,
            "used": 326832128,
            "free": 2920488960,
            "active": 530309120,
            "inactive": 345632768,
            "buffers": 138735616,
            "cached": 630099968,
            "shared": 29036544,
            "slab": 136781824
        },
        "disk": {
            "total": 2080456704,
            "used": 16752640,
            "free": 1941737472,
            "percent": 0.9
        }
    },
    "configuration": {
        "experimental": true,
        "satnogs_antenna": "RX",
        "satnogs_api_token": "[redacted]",
        "satnogs_dev_args": "biastee=true",
        "satnogs_post_observation_script": "/usr/local/bin/satnogs-post {{ID}} {{FREQ}} {{TLE}} {{TIMESTAMP}} {{BAUD}} {{SCRIPT_NAME}}",
        "satnogs_ppm_error": "0",
        "satnogs_pre_observation_script": "/usr/local/bin/satnogs-pre {{ID}} {{FREQ}} {{TLE}} {{TIMESTAMP}} {{BAUD}} {{SCRIPT_NAME}}",
        "satnogs_rf_gain": "29.7",
        "satnogs_rx_samp_rate": "2.048e6",
        "satnogs_soapy_rx_device": "driver=rtlsdr,serial=00001220",
        "satnogs_station_elev": "5",
        "satnogs_station_id": "2176",
        "satnogs_station_lat": "[redacted]",
        "satnogs_station_lon": "[redacted]",
        "snmpd_enabled": true,
        "snmpd_rocommunity": "[redacted]",
        "udp_dump_host": "127.0.0.1",
        "udp_dump_port": "57356"
    }
}
------------[ copy end ]-------------

Log information:

Feb 25 07:36:03 hostname satnogs-client[5361]: gr-satellites: Observation: 5533447, Norad: 39433, Name: 0_HUMSAT_D, Script: satnogs_afsk1200_ax25.py
Feb 25 07:36:03 hostname satnogs-client[5361]: gr-satellites: Starting observation 5533447
Feb 25 07:36:03 hostname satnogs-client[5361]: gr-satellites: running at 48000 sps
Feb 25 07:36:04 hostname satnogs-client[5361]: Found Rafael Micro R820T tuner
Feb 25 07:36:05 hostname satnogs-client[5361]: Found Rafael Micro R820T tuner
Feb 25 07:36:05 hostname satnogs-client[5361]: [INFO] Opening Generic RTL2832U OEM :: 00001220...
Feb 25 07:36:05 hostname satnogs-client[5361]: Found Rafael Micro R820T tuner
Feb 25 07:36:06 hostname satnogs-client[5361]: [R82XX] PLL not locked!
Feb 25 07:36:06 hostname satnogs-client[5361]: [INFO] Using format CF32.
Feb 25 07:36:06 hostname satnogs-client[5361]: Allocating 15 zero-copy buffers
Feb 25 07:36:16 hostname satnogs-client[5361]: cb transfer status: 2, canceling...
Feb 25 07:40:29 hostname satnogs-client[5361]: Resetting device...
Feb 25 07:40:30 hostname satnogs-client[5361]: gr-satellites: Observation: 5533447, Norad: 39433, Name: 0_HUMSAT_D, Script: satnogs_afsk1200_ax25.py
Feb 25 07:40:30 hostname satnogs-client[5361]: gr-satellites: Stopping observation 5533447
Feb 25 07:40:30 hostname satnogs-client[5361]: satnogsclient.observer.observer - ERROR - Waterfall data array is empty
Feb 25 07:40:31 hostname satnogs-client[5361]: apscheduler.executors.default - ERROR - Job "spawn_observer (trigger: date[2022-02-25 07:36:03 UTC], next run at: 2022-02-25 07:36:03 UTC)" raised an exception
Feb 25 07:40:31 hostname satnogs-client[5361]: Traceback (most recent call last):
Feb 25 07:40:31 hostname satnogs-client[5361]:   File "/var/lib/satnogs/lib/python3.7/site-packages/apscheduler/executors/base.py", line 125, in run_job
Feb 25 07:40:31 hostname satnogs-client[5361]:     retval = job.func(*job.args, **job.kwargs)
Feb 25 07:40:31 hostname satnogs-client[5361]:   File "/var/lib/satnogs/lib/python3.7/site-packages/satnogsclient/scheduler/tasks.py", line 65, in spawn_observer
Feb 25 07:40:31 hostname satnogs-client[5361]:     observer.observe()
Feb 25 07:40:31 hostname satnogs-client[5361]:   File "/var/lib/satnogs/lib/python3.7/site-packages/satnogsclient/observer/observer.py", line 261, in observe
Feb 25 07:40:31 hostname satnogs-client[5361]:     artifact = Artifacts(waterfall, metadata)
Feb 25 07:40:31 hostname satnogs-client[5361]:   File "/var/lib/satnogs/lib/python3.7/site-packages/satnogsclient/artifacts.py", line 30, in __init__
Feb 25 07:40:31 hostname satnogs-client[5361]:     self._waterfall_data = waterfall.data
Feb 25 07:40:31 hostname satnogs-client[5361]: AttributeError: 'NoneType' object has no attribute 'data'
Feb 25 08:17:12 hostname satnogs-client[5361]: satnogsclient.scheduler.tasks - ERROR - Observer job lock acquiring timed out.
Feb 25 08:44:42 hostname satnogs-client[5361]: satnogsclient.scheduler.tasks - ERROR - Observer job lock acquiring timed out.
Feb 25 08:50:22 hostname satnogs-client[5361]: satnogsclient.scheduler.tasks - ERROR - Observer job lock acquiring timed out.
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information