Create integration tests

This is a proposal regarding issue #131 (closed)

Sooooo what's in there :

The integration tests for Polaris are using the robotframework in order to simulate divers polaris process and than check the results.

All the integration tests machinery is in the new robot folder. Inside this folder you will find

• A libraries folder containing a library for comparing files. (this folder is used to add custom code to extend the robotframework for our needs).
• A testsuites folder that contains the robot source file that defines our integration tests. At this time there is only one, but of course it can be changed, i don t think that is necessary for now but when we will have more tests for sure.
• A resources folder containing all the files needed for running the tests and for testing the results.

The gitlab-ci.yml has been modified to run the integration tests and at the end of that step you will be able to download artifacts (the output folder produced by robotframework containing logs, ... and the gl.log file produced by glouton that i found very helpful during the implementation, but can be removed if you think so)

Of course the tox.ini file has been modified to add the integration tests via a robot command.

The resources folder :

• check_files contains the files that will be needed to compare the files generated during test to ensure the results are correct.
• demoddata_csv contains the csv file that will be dropped during the fetch process using the download method in order to mock a real download and avoiding making requests to the satnogs API.
• *.json are the files used to run the polaris batch command
• *.csv is the file used for the polaris fetch via import process.

The implemented integration tests covers :

• polaris fetch via import and download (all the calls to satnogs has been mocked)
• polaris learn without using gridsearch (be aware that the force cpu option must be set, if not the test is taking too much time for the gitlab ci)
• polaris batch via config file (only the learn process is started, because the rest of the polaris process are not yet able to run with config files)
• polaris viz

Result :

You can see it here

Notes :

• The gridsearch method is not tested because i will take too much time on the gitlab ci. That said we can do something like a local-robot tox command that will be used only locally and run manually or implement something like a night build. But this can be made in another MR, i think this one is already good for a first implementation.

• I shrinked the csv file containing all the lightsail-2 frames to avoid the test being too long.

• All the learn test are running using the force cpu option again to avoid the gitlab ci timout of 1 hour (and imho i wouldn t want the build running for 1 hour each time a push is made).

• The comparison for the normalized frame file is made on the size and couldn't be made on the content, because the order of the data made during the merge between the frames and the space weather is random, so the test will fail almost every time.

• The csv files have been moved to git LFS because there are just static resources and don't need to be saved in the repository (take space, no need to be versioned).

Bug fix :

During the implementation of the test, thanks to those tests i found a bug regarding the force cpu option used with config file and fixed it. The fix is in a separated commit.