Offload memory into file per clusternumber and samplesize
Created by: vcaballero-salle
Currently, all experiments until max clusternumber and simplesize are saved into memory; then, results are written in a file. This causes memory overflows for large number of experiments/iterations. We should create a file at lest per clusternumber and samplisize. Maybe by a batch of iterations too. To do that:
-
Generate all settings first. -
Loop throgh the list of settings and run the experiment. -
Per each run experiment, write a file. -
Make sure that memory can be released after writting the file, both for settings and for experiment results.