Cannot install project using poetry

Installing the project is a pain because of dependencies.

• Historically, some packages (numpy, scipy, opencv) won't install from source, or will take forever to compile but usually fail.
• piwheels fixed this but we often had to pin very specific versions to get it to work
• poetry is not good at distinguishing between different platforms

We now have a situation where installing the dependencies works using pip but installing the same dependencies with poetry is both painfully slow and often fails on either the Pi or a non-Pi platform (i.e. the CI).

I thought I had a solution: if I add a custom package source, and point it at piwheels, this causes poetry to prefer piwheels and find the right versions of the compiled wheels. However, this breaks installation on the CI (though not on Windows?!).

Setting said custom source to be "secondary" did not help.

Adding a custom source that was actually pypi helped a bit (markupsafe 1.1.1 did install) but didn't fix all the packages.

Removing the custom source installs fine on Linux (WSL Ubuntu 20.4 and Manjaro), but the lockfile it produces fails on the Pi because typed-ast won't compile. pip install typed-ast completes without error using piwheels, and finds exactly the same version poetry was trying to install (1.4.2).

As an experiment, I generated a requirements.txt from my linux install (i.e. without the piwheels source) using pip list --format=freeze and used that to install on the Pi. With the exception of numpy-stubs (which is no longer needed as it's part of numpy 1.20.1), this installed straight away. After doing that, poetry install worked without errors (because the packages were already installed). Interestingly, poetry was able to install numpy-stubs which is very strange!

Are we tying ourselves in knots for no good reason with poetry? I feel we might be.