Skip to content

CIP-2003 enhance system tests to handle multiple delay models

marking as draft since the speedrun test is also failing at the last two histograms - looking into this

Note: the delay_models_formatter and the delay_models_publisher are temporarily included in this MR because they have some changes that are not in the latest ec version (there is a separate MR on the ec side), so they do not be to be reviewed as part of this MR

Description

  • delay models publisher (from CIP-1620) called asynchronously, so that the test can continue with other tasks while delay models are being published
    • Drop delay models (by receiving delay_model_config as a JSON, see comment in SP-4061)
    • Updates cadence(s) (by receiving delay_model_config as a JSON, see comment in SP-4061)
  • new system test added for multiple delay models
    • TODO: switch the talon used to talon3 (temporarily using whichever one is available for testing)
  • delay models used for testing updated as per design in SP-4061, including:
    • delay_model_package.json converted to delay_model_config.json (system-tests and internal-schemas)
    • delay models moved from delay_model_package.json into standalone files (system-tests and internal-schemas)
  • system-tests will need to save the test parameters. The modified delay models should be saved, instead of just the files prior to modifications.
    • TODO: filter out the unused receptors on the system-tests side (this happens on the ec side and is not saved to a file)
  • Visibilities checkpoint criteria pass (zip_file_exists, fraction_of_data, number_of_timestamps (depends on CIP-2479), number_of_expected_baselines)

Testing

most recent signal chain test results:

Test_14 (signal chain verification with multiple delay models) results:

other related tests (write delay models to the TMC emulator):

other tests:

TODO: update changelog, update branch with latest main, verify other tests pass off test and signal chain tests pass. two scans does not pass (has 2 bug tracking this on JIRA)

Edited by Edna Sanchez

Merge request reports