Issue when using importlib_metadata inside a standalone application

I'm using PyInstaller to build standalone Python applications, but I've run into issues with importlib_metadata. Specfically, if I make a Python script called e.g. test_app.py with:

import importlib_metadata

then run PyInstaller with:

pyinstaller --onefile test_app.py 

The resulting application crashes when trying to get the version for PyInstaller:

$ dist/test_app 
Traceback (most recent call last):
  File "test_app.py", line 1, in <module>
    import importlib_metadata
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "/Users/tom/python/numpy-only/lib/python3.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 627, in exec_module
    exec(bytecode, module.__dict__)
  File "importlib_metadata/__init__.py", line 471, in <module>
  File "importlib_metadata/__init__.py", line 438, in version
  File "importlib_metadata/__init__.py", line 411, in distribution
  File "importlib_metadata/__init__.py", line 184, in from_name
importlib_metadata.PackageNotFoundError: importlib_metadata
[12943] Failed to execute script test_app

Other users of PyInstaller seem to have run into this too (see e.g. this issue).

I've only ever run into this issue with importlib_metadata, and haven't had a chance to investigate what this could be due to.