Skip to content

Make the script make_image_data reproducible. Some small bug fixes.

Jakob Schiøtz requested to merge schiotz/NeuralNetwork_HRTEM:reproducible-images into master

Changes:

  • The random seed is now read from the parameter.json file if present, and added to the output parameter.json file.

  • MakeImages now uses the max allowed number of cpu cores regardless of batch system.

  • Max number of cpus can be set at command line (to prevent oversubscribe login nodes used for testing). This does not affect reproducibility.

  • Buglet fix: A few places depended on the folder name ending with a slash.

  • Buglet fix: args.train does not reflect the --train argument, as that is stored in the args.test attibute, but happens to exist and always be True.

  • Batch size is adaptive for better load balancing on many-core machines: If there are many cores the batch size is now a multiple (5) of the number of cores, for better load balancing. Avoiding the case of generating 100 images on 40 cores, leading to half the cores making 2 and half making 3 images.

  • out_dir and data_dir were two names for the same folder (needlessly confusing).

Merge request reports