Cannot install project using poetry
Installing the project is a pain because of dependencies.
- Historically, some packages (
numpy
,scipy
,opencv
) won't install from source, or will take forever to compile but usually fail. -
piwheels
fixed this but we often had to pin very specific versions to get it to work -
poetry
is not good at distinguishing between different platforms
We now have a situation where installing the dependencies works using pip
but installing the same dependencies with poetry
is both painfully slow and often fails on either the Pi or a non-Pi platform (i.e. the CI).
I thought I had a solution: if I add a custom package source, and point it at piwheels, this causes poetry to prefer piwheels and find the right versions of the compiled wheels. However, this breaks installation on the CI (though not on Windows?!).
Setting said custom source to be "secondary" did not help.
Adding a custom source that was actually pypi helped a bit (markupsafe
1.1.1 did install) but didn't fix all the packages.
Removing the custom source installs fine on Linux (WSL Ubuntu 20.4 and Manjaro), but the lockfile it produces fails on the Pi because typed-ast
won't compile. pip install typed-ast
completes without error using piwheels, and finds exactly the same version poetry was trying to install (1.4.2
).
As an experiment, I generated a requirements.txt
from my linux install (i.e. without the piwheels source) using pip list --format=freeze
and used that to install on the Pi. With the exception of numpy-stubs
(which is no longer needed as it's part of numpy
1.20.1), this installed straight away. After doing that, poetry install
worked without errors (because the packages were already installed). Interestingly, poetry
was able to install numpy-stubs
which is very strange!
Are we tying ourselves in knots for no good reason with poetry
? I feel we might be.