Skip to content

Estimating predictions quality and expose that to the user

This is an MR regarding the issue #67 (closed) .

Regarding the estimation predictions changes :

  • Data are splited to train and test dataset (80%-20%).
  • Only the train dataset is passed to the fit method.
  • We make predictions.
  • We compute the quality using RMSE.
  • The results are displayed during the process using the LOGGER.

Regarding the exposition to the user:

  • I added the mlflow package to the solution.
  • I used the logging functions to expose the results of the predictions.
    1. The log_param is used for logging the context of the flow (used model, hyper parameters, ...)
    2. The log_metric is used for the results.
    3. The set_experiment is used to create an experiment with the data source name (sat name for the json file and file name if the data are from a csv file)
    4. The start_run is used to name the actual run to cross_correlate because it s the ML process running.

mlflow4

mlflow5

mlflow6

WIP or Not WIP ?

Well it depends if i have to do more configuration on mlflow in this MR. Because yes Mlflow need to be fine tuned, i'm thinking for example :

  • Making a good use of the experiment feature.
  • The place where the mlflow's db is created need to be well configured, at this time, the folder is created from where Polaris is running. I think the best way to do that is using the configuration file created by @saintaardvark .
  • We need to find a way to configure mlflow also to use a external server (deepchaos) for making results available online for testing, demo, ...

I would be really happy to do that, but does it has to be covered by this MR ? in my opinion, it's a little bit out of topic but, it's a team call :p

Edited by deck

Merge request reports

Loading