Skip to content

Outsource utopya

Utopia Developers requested to merge outsource-utopya into master

What does this MR do?

This MR removes the utopya implementation from this repository in favour of an installation of the (more maintainable and generalised) outsourced version of utopya, further developed here.

Of course, this is a major change to the repository structure – but it only incurs minor changes to using Utopia.

Overview of interface changes

  • CLI changes:
    • --no-plot renamed to --no-eval
    • --sweep and --single replaced by --run-mode {single, sweep}
    • --po/--plot-only takes only a single argument, but can be given multiple times: --po some_plot --po some_other_plot
  • Changes to model tests and plots: The utopya and dantro package structure has changed, imports may need to be adapted.

For separate models repositories, some minor adaptations need to be made:

  • Rename the Cmake call to register_models_with_frontend to register_models_with_utopya
  • Add a project information file .utopya-project.yml to the root directory of the project, content akin to this

Is there something that needs to be double checked?

  • Downloading and installing Utopia with outsourced utopya works as described in README:
    • Checked via CI (Ubuntu 20.04)
    • Checked by @tgaskin on macOS with Apple Silicon
    • Checked by @blsqr on macOS with Apple Silicon
    • Checked by @mackharald89 on macOS with Apple Intel
    • Checked by @herdeanu on macOS with Apple Intel
    • Checked by @peanutfun on Ubuntu 20.04 (follow-up issue from this remark necessary)
  • README text about versioning ok?
  • Upgrade process for downstream Utopia projects work as expected. (Follow instructions from README.)
  • (Optional) Suggestions for the expanded plot documentation (added in !280 (merged))? See review/docs environment for the latest deployment ("View App" button under CI/CD status indicators), and navigate to Evaluating Simulations → Plotting in the sidebar

Can this MR be accepted?

  • Outsource utopya
    • Let cmake install utopya
    • Adjust cmake call to utopya CLI
    • Add symlink utopia -> utopya to make interface backwards-compatible (and retain muscle memory)
    • Remove python/utopya (history retained in outsourced utopya project)
    • Adapt to new project and model registration procedure
    • Adapt to interface changes
    • Check that downstream Utopia projects remain working
      • ... with changes, see e.g. here (not all changes necessary for migration)
  • Update remaining python code
    • Fix importing issues in plots
    • ❗ Fix/migrate/modernize plots and re-activate disabled ones
  • Improve repository infrastructure (to improve maintainability)
    • Let cmake install pre-commit
    • Add pre-commit configuration
    • Apply black code formatting to remaining python code
    • Add some hooks that should help to keep files tidy
      • ... can also extend these in the future, e.g. with C++ code formatting
  • Added or adjusted tests
    • Removed test:utopya CI job
    • Adapted CI structure to allow running models in docs:build job
    • Reduced test:CLI job to the bare basics; this is all tested in utopya (which is responsible for the CLI)
    • Added a check:hooks job that lets the pre-commit hooks run (enforcing that it is run locally before pushing)
  • Updated documentation
    • Adjust and improve README
      • Inform about utopya having been outsourced
      • Inform about v1 release branch and about versioning
      • Inform how to migrate from v1 to the latest version of Utopia
    • Use intersphinx to cross-reference other packages
    • Update all the cross-references such that there are no more warnings
      • Throws out all the automodule directives in favour of linking to the place where the code is actually documented: the respective package documentations
    • Run doctest (failing on errors) and linkcheck (not failing, for visual inspection only)
    • Include utopya as a submodule to allow including tested example code
      • Of course this has to change in the long term ... more of a workaround right now
  • Ready for merging

Related issues

Closes #11 (closed), #100 (closed), #333 (closed), #224 (closed), #230 (closed), #44 (closed), #285 (closed), #349 (closed), #353 (closed) ... 🚧

Edited by Yunus Sevinchan

Merge request reports