Move away from poetry

I have moved from poetry to pipenv and setuptools. This feels like a step backwards in some ways (less integrated, more config files) but it has one massive advantage, which is that the project now installs successfully on Raspberry Pi, Windows, and Linux. The lockfile can be successfully generated on Windows and RPi. We also now don't require anything other than python and pip to install - no piping into sudo bash in the CI any more!

The new configuration is as follows:

  • Dependencies, and dev-dependencies, are specified in setup.py in the usual way (using install_requires and extra_requires[dev]).
  • Package metadata is specified in setup.py in the usual way.
  • pyproject.toml is retained, but no longer includes package metadata or dependencies and does not have a [tool.poetry] table. It does define the build system as per PEP517, which is setuptools, and it also contains settings for black, poe and isort.
  • Pipfile is very minimal, and only declares dependencies on the current module, i.e. the dependencies declared in setup.py. It also specifies the Python version. This allows us to single-source dependency information from setup.py but use the dependency resolution/locking functionality of pipenv
  • Pipfile.lock locks the dependency versions in the same way as poetry.lock.

To install the project, the steps are broadly similar, but pipenv doesn't have a handy option to put the virtual environment in the project folder, so we must do that manually:

python -m venv .venv
pip install pipenv
pipenv install --dev

I'm claiming this is actually simpler than with poetry because there's no need to install poetry first. NB the --dev is very important, otherwise you only get install dependencies. Also, when updating/upgrading, it's important to specify --dev otherwise the dev dependencies will not be updated (but will still be present with the old versions).

poe is retained as our command-manager so the usual checks/automations can be run, but rather than poetry run poe XXX you must activate the virtual environment, then run poe XXX. poe is updated to v0.10 because that was required in order for it to function without poetry.

This MR also upgrades a couple of packages - most notably numpy. Upgrading to numpy==1.20.0 removes the need for a separate numpy-stubs type information package, as that project was merged into numpy. It also deprecated numpy.float (which was always an alias for float) so I have modified the type hints in a few places to fix this.

black is now fixed to 0.18.9b0 because the "compatible" version specifier fails for prerelease packages. I don't think this should be a problem for us, it only becomes an issue when trying to install multiple packages in dev mode that use different versions of black.

This MR closes #218 (closed). That issue details some of the rejected options (like requirements.txt) so it's worth a read if you think there's an obvious solution that isn't this one.

Things that need to happen before this can be merged:

  • Move metadata and dependencies from pyproject.toml to Pipfile and setup.py
  • Update README.md with correct install instructions
  • Update the CI to use pipenv rather than poetry
  • Fix issues in the CI

Things that need to happen before this can be released:

  • Update the ofm command-line scripts so that they use the correct install method. This applies to ofm upgrade and ofm develop. I have started a merge request for this in openflexure-microscope-cli!14 (merged).
Edited by Richard Bowman

Merge request reports

Loading