CSV format for `polaris learn` not working: "KeyError: 'analysis'"
When running polaris learn
on https://gitlab.com/librespacefoundation/polaris/polaris-playground/-/blob/master/data/normalised_frames.csv, I ran into an error:
$ polaris learn --output_graph_file graph_from_csv.json ./normalised_frames.csv
/home/aardvark/dev/polaris/polaris/data/readers.py:58: ParserWarning: Falling back to the 'python' engine because the 'c' engine does not support sep=None with delim_whitespace=False; you can avoid this warning by specifying engine='python'.
dataframe = pd.read_csv(path, sep=csv_sep)
2021-05-13 08:17:01,089 - polaris.learn.predictor.cross_correlation_configurator - INFO - Plain old gridsearch parameters!
2021-05-13 08:17:01,095 - polaris.learn.predictor.cross_correlation_configurator - INFO - No GPU detected! Adding CPU parameters :)
2021-05-13 08:17:01,095 - polaris.feature.cleaner_configurator - INFO - Default thresholds used for dataset na values (col:30% row:60%)!
Traceback (most recent call last):
File "/home/aardvark/dev/polaris/.venv/bin/polaris", line 33, in <module>
sys.exit(load_entry_point('polaris-ml', 'console_scripts', 'polaris')())
File "/home/aardvark/dev/polaris/.venv/lib64/python3.8/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/home/aardvark/dev/polaris/.venv/lib64/python3.8/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/home/aardvark/dev/polaris/.venv/lib64/python3.8/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/aardvark/dev/polaris/.venv/lib64/python3.8/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/aardvark/dev/polaris/.venv/lib64/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/home/aardvark/dev/polaris/polaris/polaris.py", line 157, in cli_learn
cross_correlate(input_file,
File "/home/aardvark/dev/polaris/polaris/learn/analysis.py", line 107, in cross_correlate
xcorr = XCorr(metadata, xcorr_configurator.get_configuration())
File "/home/aardvark/dev/polaris/polaris/learn/predictor/cross_correlation.py", line 54, in __init__
if 'feature_columns' in dataset_metadata['analysis'] else []
KeyError: 'analysis'
@adithyav1511 has suggested a possible source and solution:
When we switched to the configurator, we wanted to remove space weather from being predicted. If I am correct this code is from that commit. Scratch that. This seems to be from switching to the newer data format (where we add metadata during fetch).
How to duplicate
- Clone https://gitlab.com/librespacefoundation/polaris/polaris-playground
- Run
polaris learn --output_graph_file graph_from_csv.json path_to_that_repo/data/normalized_frames.csv