Skip to content

SAR-364 Add DishLogger

Johan Venter requested to merge sar-364-Add-DishLogger into master

To enable us to get logs out of the C++ devices and into elastic we're using TLS to forward the logs from the LMC devices to our new DishLogger. DishLogger consumes these logs and logs to stdout as per the SKA standard.

This MR has 4 parts:

  • Add the DishLogger Python Tango device

    • Tests and device added
    • A new log filter added to override the log components as we get them from TLS
    • Log to stdout
    • Added pyproject.toml to enable installation

  • Update the Helm charts

    • Deploy DishLogger
    • Update log target and enable logging to it
    • DishLogger starts first to catch logs as LMC devices starts up
      • Order of starting devices:
        • DishLogger
        • LMC devices
        • DishMaster

  • Dockerfile

    • Install Python packages
    • Removed log4cxx & jsoncpp compile step as we install the headers via apt-get
    • Update the PATH so we can just run DishLogger (specified as script in pyproject.yaml)
    • Installing software from source (via COPY) and no longer cloning the repo for install

  • PDUManager

    • We never deployed it so I just added it in

TODO:

  • Need to get the pipeline sorted

Sample logs:

device-server 1|2022-05-09T13:56:28.559Z|WARNING|unknown_thread|DishBaseDevice::HandleEvent()|unknown_file#0|tango-device:mid_d0001/elt/master|One or more event handler commands failed │
│ device-server 1|2022-05-09T13:56:31.559Z|WARNING|unknown_thread|DishBaseDevice::HandleEvent()|unknown_file#0|tango-device:mid_d0001/elt/master|One or more event handler commands failed │
│ device-server 1|2022-05-09T13:56:31.559Z|ERROR|unknown_thread|DishBaseDevice::ExecuteEventCmdHandlers()|unknown_file#0|tango-device:mid_d0001/elt/master|Failed to extract data from eve │
│ device-server 1|2022-05-09T13:56:31.560Z|WARNING|unknown_thread|TangoUtils::DeviceAttributeToCorbaAny()|unknown_file#0|tango-device:mid_d0001/elt/master|FMT_UNKNOWN data format for att │
│ device-server 1|2022-05-09T13:56:32.106Z|INFO|unknown_thread|DishMaster::handle_band_in_focus_event()|unknown_file#0|tango-device:mid_d0001/elt/master|Setting SPF band in focus to 1... │
│ device-server 1|2022-05-09T13:56:34.559Z|WARNING|unknown_thread|TangoUtils::DeviceAttributeToCorbaAny()|unknown_file#0|tango-device:mid_d0001/elt/master|FMT_UNKNOWN data format for att │
│ device-server 1|2022-05-09T13:56:34.559Z|ERROR|unknown_thread|DishBaseDevice::ExecuteEventCmdHandlers()|unknown_file#0|tango-device:mid_d0001/elt/master|Failed to extract data from eve │
│ device-server 1|2022-05-09T13:56:34.560Z|WARNING|unknown_thread|DishBaseDevice::HandleEvent()|unknown_file#0|tango-device:mid_d0001/elt/master|One or more event handler commands failed │
│ device-server 1|2022-05-09T13:56:37.559Z|WARNING|unknown_thread|TangoUtils::DeviceAttributeToCorbaAny()|unknown_file#0|tango-device:mid_d0001/elt/master|FMT_UNKNOWN data format for att │
│ device-server 1|2022-05-09T13:56:37.559Z|WARNING|unknown_thread|DishBaseDevice::HandleEvent()|unknown_file#0|tango-device:mid_d0001/elt/master|One or more event handler commands failed │
│ device-server 1|2022-05-09T13:56:37.559Z|ERROR|unknown_thread|DishBaseDevice::ExecuteEventCmdHandlers()|unknown_file#0|tango-device:mid_d0001/elt/master|Failed to extract data from eve │
│ device-server 1|2022-05-09T13:56:40.558Z|ERROR|unknown_thread|DishBaseDevice::ExecuteEventCmdHandlers()|unknown_file#0|tango-device:mid_d0001/elt/master|Failed to extract data from eve │
│ device-server 1|2022-05-09T13:56:40.559Z|WARNING|unknown_thread|TangoUtils::DeviceAttributeToCorbaAny()|unknown_file#0|tango-device:mid_d0001/elt/master|FMT_UNKNOWN data format for att │
│ device-server 1|2022-05-09T13:56:40.560Z|WARNING|unknown_thread|DishBaseDevice::HandleEvent()|unknown_file#0|tango-device:mid_d0001/elt/master|One or more event handler commands failed
Edited by Samuel Twum

Merge request reports