Unable to retrieve resources from a namespace package
Attempting to retrieve resources from a namespace package fails.
draft $ mkdir foo
draft $ touch foo/bar.txt
draft $ rwt importlib_resources
Collecting importlib_resources
Using cached https://files.pythonhosted.org/packages/2f/f7/b4aa02cdd3ee7ebba375969d77c00826aa15c5db84247d23c89522dccbfa/importlib_resources-1.0.2-py2.py3-none-any.whl
Installing collected packages: importlib-resources
Successfully installed importlib-resources-1.0.2
Python 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 03:13:28)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import importlib_resources
>>> importlib_resources.read_text(__import__('foo'), 'bar.txt')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/resources.py", line 169, in read_text
with open_text(package, resource, encoding, errors) as fp:
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/resources.py", line 126, in open_text
_check_location(package)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/importlib/resources.py", line 82, in _check_location
raise FileNotFoundError(f'Package has no location {package!r}')
FileNotFoundError: Package has no location <module 'foo' (namespace)>
I see an obvious problem here - that a namespace package can have more than one base path, so it has no single location. But it does have a location... and pkg_resources lets one load resources from namespace package:
draft $ cat > foo/__init__.py
import pkg_resources; pkg_resources.declare_namespace('foo')
draft $ python
Python 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 03:13:28)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pkg_resources
>>> pkg_resources.resource_stream('foo', 'bar.txt')
<_io.BufferedReader name='/Users/jaraco/draft/foo/bar.txt'>
pkg_resources
doesn't succeed with a PEP 420 namespace package:
$ rm foo/__init__.py
>>> pkg_resources.resource_stream('foo', 'bar.txt')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1145, in resource_stream
return get_provider(package_or_requirement).get_resource_stream(
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pkg_resources/__init__.py", line 359, in get_provider
return _find_adapter(_provider_factories, loader)(module)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1387, in __init__
self.module_path = os.path.dirname(getattr(module, '__file__', ''))
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/posixpath.py", line 156, in dirname
p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType
But even in that situation, it does allow for loading resources for a module within a PEP 420 namespace package:
draft $ touch foo/mod.py
draft $ tree
.
└── foo
├── bar.txt
└── mod.py
draft $ python
Python 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 03:13:28)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pkg_resources
>>> pkg_resources.resource_stream('foo.mod', 'bar.txt')
<_io.BufferedReader name='/Users/jaraco/draft/foo/bar.txt'>
This issue sort-of relates to #60 (closed), but is more serious because it seems there's no input that importlib_resources
can accept to load resources from a namespace package.
The issue emerged in pmxbot when I tried to convert the package from a (deprecated) pkg_resources-style namespace package to a PEP 420 namespace package. The code failed at this line when it tried to load the phrases.
It seems to me (without looking at the code) it should be straightforward to support loading resources from namespace packages, following the same logic that importlib follows to import a module from that package.
The only other option I see is to force packages to rewrite their packages to only put resources in non-namespace packages, which is a bit of an imposition and unintuitive constraint.