Skip to content

Resolve "Add basic error handling to transformation framework"

Utopia Developers requested to merge 264-dag-error-handling into master

This MR adds error handling to the data transformation framework (#264 (closed)) and integrates it into the MultiversePlotCreator to allow data selection with missing data (#256 (closed)).

Example: Failing data operations

  transform:
    - float: "inf"
    - div: [1, 0]               # --> ZeroDivisionError
      allow_failure: silent     # can also be: True, log, warn, False
      fallback: !dag_prev       # will use this as result (can also be any other tag, scalar, mapping, sequence, ...)

... but also works with upstream errors ... see updated docs page for more info.

Example: Plotting with missing data

my_plot:
  # Select data, allowing for missing universes or failing .mean operation
  select_and_combine:
    allow_missing_or_failing: true
    combination_method: merge        # needed with allow_missing_or_failing
    fields:
      data:
        path: randints
        transform:
          - .mean: [!dag_prev , [x]]

Anything to double-check?

  • Docs understandable?
  • Usage ok?

Can this MR be accepted?

  • Code quality
  • Ready for merging
    • Pipeline passes without warnings
    • History cleaned-up or squash option set
    • Changelog entry added
    • Version number bumped
    • Reviewed & approved

Related issues

Closes #264 (closed), #256 (closed)

Edited by Utopia Developers

Merge request reports