CI: Caching outside repository seems impossible

Apparently I cannot upload any cache of, say, /usr/local/lib/python3.5/site-packages, where the packages are stored when installing dependencies on the python:3 container.