Towards deploying on Linux

This issue is just to track some of the roadblocks and work-arounds to make deployment on Linux possible.

Problem 1: QiTissue is deployed on Linux as an AppImage package which get unpacked into a runnable environment every time upon starting the AppImage. Thus any symbolic links that prepackaged-qi-integration makes to QiTissue's python and in-QiTissue files has to be re-done every time. Work-around: the AppImage can be unpacked permanently using ./QiTissue.... --appimage-extract That creates a squashfs-root directory which can be renamed to QiTissue and QiTissue can be started by running QiTissue/AppRun

Problem 2: running install_stardist fails with error:

import importlib.util
spec = importlib.util.spec_from_file_location("install_stardist", "/home/dsudar/Documents/QiScripts/prepackaged_integration/install_stardist.py")
install_stardist = importlib.util.module_from_spec(spec)
spec.loader.exec_module(install_stardist)

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-1-099612999b50> in <module>
      2 spec = importlib.util.spec_from_file_location("install_stardist", "/home/dsudar/Documents/QiScripts/prepackaged_integration/install_stardist.py")
      3 install_stardist = importlib.util.module_from_spec(spec)
----> 4 spec.loader.exec_module(install_stardist)

~/QiTissue/usr/lib/python3.9/importlib/_bootstrap_external.py in exec_module(self, module)

~/QiTissue/usr/lib/python3.9/importlib/_bootstrap.py in _call_with_frames_removed(f, *args, **kwds)

~/Documents/QiScripts/prepackaged_integration/install_stardist.py in <module>
     15     change_to_module_cwd()
     16     logging.debug('Changed working dir')
---> 17     create_and_install_package('prepacked_stardist==2.0.0',
     18                                'https://gitlab.com/api/v4/groups/claiv/-/packages/pypi/simple')
     19     logging.info('Installation prepackaged_stardist finished')

~/Documents/QiScripts/prepackaged_integration/_utils/install_in_venv.py in create_and_install_package(package_name, extra_index_url)
     21     venv_dir = get_venv_name(package_name)
     22 
---> 23     create_venv(venv_dir)
     24     logging.debug('activating virtual env')
     25     activate_virtual_environment(venv_dir)

~/Documents/QiScripts/prepackaged_integration/_utils/install_in_venv.py in create_venv(venv_dir)
     31 
     32 def create_venv(venv_dir: str):
---> 33     cli_run(["-p", get_python_executable(), venv_dir], setup_logging=False)
     34 
     35 

~/QiTissue/usr/lib/python3.9/dist-packages/virtualenv/run/__init__.py in cli_run(args, options, setup_logging, env)
     31     of_session = session_via_cli(args, options, setup_logging, env)
     32     with of_session:
---> 33         of_session.run()
     34     return of_session
     35 

~/QiTissue/usr/lib/python3.9/dist-packages/virtualenv/run/session.py in run(self)
     43     def run(self):
     44         self._create()
---> 45         self._seed()
     46         self._activate()
     47         self.creator.pyenv_cfg.write()

~/QiTissue/usr/lib/python3.9/dist-packages/virtualenv/run/session.py in _seed(self)
     56         if self.seeder is not None and self.seeder.enabled:
     57             logging.info("add seed packages via %s", self.seeder)
---> 58             self.seeder.run(self.creator)
     59 
     60     def _activate(self):

~/QiTissue/usr/lib/python3.9/dist-packages/virtualenv/seed/embed/via_app_data/via_app_data.py in run(self, creator)
     40         if not self.enabled:
     41             return
---> 42         with self._get_seed_wheels(creator) as name_to_whl:
     43             pip_version = name_to_whl["pip"].version_tuple if "pip" in name_to_whl else None
     44             installer_class = self.installer_class(pip_version)

~/QiTissue/usr/lib/python3.9/contextlib.py in __enter__(self)
    117         del self.args, self.kwds, self.func
    118         try:
--> 119             return next(self.gen)
    120         except StopIteration:
    121             raise RuntimeError("generator didn't yield") from None

~/QiTissue/usr/lib/python3.9/dist-packages/virtualenv/seed/embed/via_app_data/via_app_data.py in _get_seed_wheels(self, creator)
    126         if fail:
    127             msg = f"seed failed due to failing to download wheels {', '.join(fail.keys())}"
--> 128             raise RuntimeError(msg)
    129         yield name_to_whl
    130 

RuntimeError: seed failed due to failing to download wheels pip, wheel, setuptools

Is this the problem that Yannick referred to that distutils is not included in QiTissue's Python environment? When I ask the QiTissue Python interpreter to list available modules, it gives this:

help('modules')

Please wait a moment while I gather a list of all available modules...

/home/dsudar/QiTissue/usr/lib/python3.9/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
/home/dsudar/QiTissue/usr/lib/python3.9/site-packages/_distutils_hack/__init__.py:18: UserWarning: Distutils was imported before Setuptools, but importing Setuptools also replaces the `distutils` module in `sys.modules`. This may lead to undesirable behaviors or errors. To avoid these issues, avoid using distutils directly, ensure that setuptools is installed in the traditional way (e.g. not an editable install), and/or make sure that setuptools is always imported before distutils.
  warnings.warn(
/home/dsudar/QiTissue/usr/lib/python3.9/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
/home/dsudar/QiTissue/usr/lib/python3.9/dist-packages/numba/tests/__init__.py:23: UserWarning: Failed to enable faulthandler due to:
fileno
  warnings.warn(msg.format(err=e))
/home/dsudar/QiTissue/usr/lib/python3.9/site-packages/_distutils_hack/__init__.py:18: UserWarning: Distutils was imported before Setuptools, but importing Setuptools also replaces the `distutils` module in `sys.modules`. This may lead to undesirable behaviors or errors. To avoid these issues, avoid using distutils directly, ensure that setuptools is installed in the traditional way (e.g. not an editable install), and/or make sure that setuptools is always imported before distutils.
  warnings.warn(
/home/dsudar/QiTissue/usr/lib/python3.9/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
<string>:121: UserWarning: The numpy.array_api submodule is still experimental. See NEP 47.
/home/dsudar/QiTissue/usr/lib/python3.9/site-packages/_distutils_hack/__init__.py:18: UserWarning: Distutils was imported before Setuptools, but importing Setuptools also replaces the `distutils` module in `sys.modules`. This may lead to undesirable behaviors or errors. To avoid these issues, avoid using distutils directly, ensure that setuptools is installed in the traditional way (e.g. not an editable install), and/or make sure that setuptools is always imported before distutils.
  warnings.warn(
/home/dsudar/QiTissue/usr/lib/python3.9/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")

The logfile.log (attached) also has a huge list of log messages.