Simplified YAML configuration files

Currently, the YAML configuration files are too versatile.

It is possible to write a YAML configuration that doesn't make sense and that will not generate an error message.

For example with this file:

simulation:
  mode: single                  # <== This is a 'single' pipeline

  parametric_mode: sequential   # <== This is specific of a 'parametric' pipeline
  parameters: ...               #     It will be not used for a 'single' pipeline

  outputs:
    output_folder:    ...
    calibration_plot: ...       # <= This is specific of a 'calibration' pipeline
                                #    It will be not used for a 'single' pipeline

Using a unique YAML structure for each modes (single, parametric, calibration...) would have the following advantages:

• simplify the code to parse a YAML file. An error can be easily spotted (e.g. you cannot use a parametric plot with a 'single' pipeline)
• remove redundant information in the YAML files (see below)
• use a better Python object to represent a configuration instead of a dict object

Example for a 'parametric' simulation

The following configuration for 'parametric' simulation contains 3 times the keyword 'parametric':

simulation:

  mode: parametric    # <== This keyword is used to define the mode of simulation

  parametric:         # <== again the keyword 'parametric'. Furthermore, it has to be
                      #     'parametric' otherwise it would not work. 
    parametric_mode: sequential
    parameters:
      - key: pipeline.photon_generation.illumination.arguments.level
        values: numpy.unique(numpy.logspace(0, 6, 100, dtype=int))

  outputs:
    output_folder:  'examples/PTC/output'
    parametric_plot:                    # <== again the keyword 'parametric'
                                        #     It cannot be 'calibration_plot'
      x:  'detector.image.mean'
      y:  'detector.image.std_deviation'
      plot_args:
        title: 'CCD Photon Transfer Curve'
        xscale: 'log'
        yscale: 'log'
        xlim: [1., 2.e+6]
        ylim: [1., 3.e+4]
        grid: true

ccd_detector: ...

pipeline: ...

Here is a proposed simplified version for a 'parametric' simulation :

parametric:    # <== keyword 'parametric' appears only once.

  mode: sequential    # <== this is a parametric's mode
  parameters:  
    - key: pipeline.photon_generation.illumination.arguments.level
      values: numpy.unique(numpy.logspace(0, 6, 100, dtype=int))

  outputs:
    output_folder:  'examples/PTC/output'
    plot:             # <== no need to state that this plot is used
                      #     for the 'parameteric' mode
      x:  'detector.image.mean'
      y:  'detector.image.std_deviation'
      args:
        title: 'CCD Photon Transfer Curve'
        xscale: 'log'
        yscale: 'log'
        xlim: [1., 2.e+6]
        ylim: [1., 3.e+4]
        grid: true

ccd_detector: ...

pipeline: ...

Example for a 'single' simulation.

Now:

simulation:
  mode: single      # <== This keyword is used to define the mode of simulation

  outputs:
    output_folder: 'examples/basic/output'
    save_data_to_file:
      -
        detector.image.array: fits
    single_plot:                  # <== again the keyword 'single'.
      y: 'detector.image.array'   #     It cannot be for example a `parametric_plot`
      plot_type: 'histogram'
      plot_args:
        bins: 200
        xlabel: 'ADU'
        yscale: 'log'
        title:  'image histogram'

After:

single:     # <== keyword 'single' appears only once.

  outputs:
    output_folder: 'examples/basic/output'
    save_data_to_file:
      -
        detector.image.array: fits
    plot:                         # We already know that it's a single plot
      y: 'detector.image.array'
      plot_type: 'histogram'
      plot_args:
        bins: 200
        xlabel: 'ADU'
        yscale: 'log'
        title:  'image histogram'

Example for a 'calibration' simulation.

Before:

simulation:

  mode: calibration

  calibration:
    calibration_mode:     pipeline
    result_type:          image
    result_fit_range:     [0, 100, 0, 100]
    target_data_path:     ['CTI/input_data/cti_image/image_irradiated.fits']
    target_fit_range:     [0, 100, 0, 100]
    seed:                 60336
    fitness_function:
      func:               pyxel.calibration.fitness.sum_of_abs_residuals
      arguments:
    algorithm:
      type:               sade
      generations:        50
      population_size:    200
      variant:            2
    parameters:
      - key:              pipeline.charge_transfer.cdm.arguments.tr_p
        values:           [_, _]
        logarithmic:      true
        boundaries:       [1.e-3, 1.e-1]
      - key:              pipeline.charge_transfer.cdm.arguments.nt_p
        values:           [_, _]
        logarithmic:      true
        boundaries:       [1.e+2, 1.e+4]

  outputs:
    output_folder:        'examples/CTI/output'
    calibration_plot:
      champions_plot:
        plot_args:
          yscale: 'log'
      population_plot:
        columns: [2, 4]
        plot_args:
          xscale: 'log'
          yscale: 'log'
          title: 'trap specie #1'
          xlabel: 'release time (s)'
          ylabel: 'density (cm$^{-3}$)'

After:

calibration:
  mode:     pipeline
  result:
    type:          image
    fit_range:     [0, 100, 0, 100]
  target:
    data_path:     ['CTI/input_data/cti_image/image_irradiated.fits']
    fit_range:     [0, 100, 0, 100]
  seed:                 60336
  fitness:
    func:               pyxel.calibration.fitness.sum_of_abs_residuals
    arguments:
  algorithm:
    type:               sade
    generations:        50
    population_size:    200
    variant:            2
  parameters:
    - key:              pipeline.charge_transfer.cdm.arguments.tr_p
      values:           [_, _]
      logarithmic:      true
      boundaries:       [1.e-3, 1.e-1]
    - key:              pipeline.charge_transfer.cdm.arguments.nt_p
      values:           [_, _]
      logarithmic:      true
      boundaries:       [1.e+2, 1.e+4]

  outputs:
    output_folder:        'examples/CTI/output'
    calibration_plot:
      champions_plot:
        plot_args:
          yscale: 'log'
      population_plot:
        columns: [2, 4]
        plot_args:
          xscale: 'log'
          yscale: 'log'
          title: 'trap specie #1'
          xlabel: 'release time (s)'
          ylabel: 'density (cm$^{-3}$)'