python USER_SITE directory is broken
Context
I often add helper tools to my flatpak containers, and some of those tools are written in python. Setting them up is normally as easy as running pip or setup.py with the --user option, which installs them under ~/.local . It worked fine in org.freedesktop.Platform 21.08.
Description
With the arrival of org.freedesktop.Platform 22.08, user-installed python packages are broken: They correctly install under ~/.local, but once installed, python cannot find them.
Investigation reveals that Freedesktop 22.08 has somehow configured python's USER_SITE to be a directory under ~/.local in some cases, and a different directory under /var in other cases. Of course, /var is not typically a suitable place for user-installed files, and even if it was, the inconsistency seems to cause the breakage.
I'll demonstrate the problem by trying to install pip, but it could just as well be any package:
$ python3 -m pip
/usr/bin/python3: No module named pip
# Okay, that's expected. Let's install it:
$ python3 -m ensurepip --user
Looking in links: /tmp/tmp2n0qwkn5
Requirement already satisfied: setuptools in /usr/lib/python3.10/site-packages (65.3.0)
Processing /tmp/tmp2n0qwkn5/pip-22.2.1-py3-none-any.whl
Installing collected packages: pip
WARNING: The scripts pip3 and pip3.10 are installed in '/home/myuser/.local/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pip-22.2.1
# Great. Now let's make sure it works:
$ python3 -m pip
/usr/bin/python3: No module named pip
# Uh... what? Didn't we just install it?
$ ls ~/.local/lib/python3.10/site-packages/
pip pip-22.2.1.dist-info
# Yep, it's there. Because python put it there. So why doesn't python see it?
$ python3 -m site --user-site
/home/myuser/.local/lib/python3.10/site-packages
# Okay, so what's the problem? Is python's user-site support enabled?
$ python3 -m site
sys.path = [
'/home/myuser',
'/usr/lib/python310.zip',
'/usr/lib/python3.10',
'/usr/lib/python3.10/lib-dynload',
'/app/lib/python3.10/site-packages',
'/usr/lib/python3.10/site-packages',
'/home/myuser/.local/lib/python3.10/site-packages',
]
USER_BASE: '/home/myuser/.local' (exists)
USER_SITE: '/home/myuser/.local/lib/python3.10/site-packages' (exists)
ENABLE_USER_SITE: True
# Yep. It's enabled, and our package is in a valid place. Something is broken.
$ python3 -c 'import site; print(site.USER_SITE)'
/var/data/python/lib/python3.10/site-packages
# Um... what? How in the world did that happen?