Append data as list of dict in DataContainer
With the upcoming eviction of occupations
from the pandas dataframe we need to think how to save runtime data in a consistent way.
An ideal solution would not include a separate list of mctrial values, which will be the case if we keep appending data to pandas dataframe.
It has been proposed by @erikfransson to save data as a list of dictionaries:
_data_rows = [{'mctrial': 0, 'occupations': [13, 11, ..], 'potential': -1.370203},
{'mctrial': 10, 'potential': -1.370203},
...]
The above list can be transformed easily into a pandas dataframe by doing:
data = pandas.DataFrame(_data_rows)
which will be provided to the user (without occcupations
) and used internally by get_data
.
Of course, it should be specified somewhere that the provided dataframe only includes scalar data.
List of Dict to Pandas Dataframe
Occupations can be excluded easily during the conversion:
data = pd.DataFrame.from_records(_data_rows, exclude='occupations')
@erhart @erikfransson @rmagnus @angqvist This suggestion is open for discussion, criticisms, likes, reactions, and so on
Updated: @williamz 05/12/2018 (Remember me Paul)
During IRL scrum meeting it has been decided to proceed forward according to the above description.