segmentation fault using python unittest module in a conda environment
Hi. I've googled for a few hours and got nothing useful. I also tried the GitLab forum but in vain. If this is a wrong place to ask, please let me know.
I'm new to GitLab. We're trying to migrate from GitHub to GitLab. Now I'm trying to run a basic CI on GitLab, but got a segmentation fault. And there seemed no detailed log to debug or investigate.
On our local Ubuntu devices, we run python -m unittest discoverin a conda environment to test the code. So I think I can do it on GitLab, too. I have another small open project that do the same thing(same .gitlab-ci.tml, same conda env file, with less actual python test code), and CI for it is successful. But a CI on the project that in a private group failed.
Here's the ".gitlab-ci.yml" file:
# Use latest conda docker image
image: python:3.6.11-stretch
before_script:
# install conda
- apt update
- wget -q https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh
- bash miniconda.sh -b -p $HOME/miniconda
- source "$HOME/miniconda/etc/profile.d/conda.sh"
- hash -r
# setup conda
- conda config --set always_yes yes --set changeps1 no
- conda update -q conda
- conda info -a
# create conda environment and activate it
- conda env create -q -n omni-py36-tf112 -f conda/omni-py36-tf112.yml
- conda activate omni-py36-tf112
test:
script:
# try to debug
- free -h
- ulimit -s
# run unittest
- python -m unittest discoverAnd here's the full CI log:
Running with gitlab-runner 13.2.0-rc2 (45f2b4ec)
on docker-auto-scale 0277ea0f
Preparing the "docker+machine" executor
00:39
Using Docker executor with image python:3.6.11-stretch ...
Pulling docker image python:3.6.11-stretch ...
Using docker image sha256:b99987202f4ea0979b4bb673494d471042956f38652f7a8faa15576e3651098a for python:3.6.11-stretch ...
Preparing environment
00:04
Running on runner-0277ea0f-project-20110274-concurrent-0 via runner-0277ea0f-srm-1595491756-eb97efe6...
Getting source from Git repository
00:19
$ eval "$CI_PRE_CLONE_SCRIPT"
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/omnieyes_general/OmniEyes/.git/
Created fresh repository.
Checking out 1f7ba67f as setup_gitlab_ci...
Skipping Git submodules setup
Executing "step_script" stage of the job script
03:41
$ apt update
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
Ign:1 http://deb.debian.org/debian stretch InRelease
Get:2 http://security.debian.org/debian-security stretch/updates InRelease [53.0 kB]
Get:3 http://deb.debian.org/debian stretch-updates InRelease [93.6 kB]
Get:4 http://deb.debian.org/debian stretch Release [118 kB]
Get:5 http://deb.debian.org/debian stretch Release.gpg [2410 B]
Get:6 http://security.debian.org/debian-security stretch/updates/main amd64 Packages [537 kB]
Get:7 http://deb.debian.org/debian stretch-updates/main amd64 Packages [2596 B]
Get:8 http://deb.debian.org/debian stretch/main amd64 Packages [7080 kB]
Fetched 7886 kB in 1s (3982 kB/s)
Reading package lists...
Building dependency tree...
Reading state information...
4 packages can be upgraded. Run 'apt list --upgradable' to see them.
$ wget -q https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh
$ bash miniconda.sh -b -p $HOME/miniconda
PREFIX=/root/miniconda
Unpacking payload ...
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... done
## Package Plan ##
environment location: /root/miniconda
added / updated specs:
- _libgcc_mutex==0.1=main
- ca-certificates==2020.1.1=0
- certifi==2020.4.5.1=py37_0
- cffi==1.14.0=py37he30daa8_1
- chardet==3.0.4=py37_1003
- conda-package-handling==1.6.1=py37h7b6447c_0
- conda==4.8.3=py37_0
- cryptography==2.9.2=py37h1ba5d50_0
- idna==2.9=py_1
- ld_impl_linux-64==2.33.1=h53a641e_7
- libedit==3.1.20181209=hc058e9b_0
- libffi==3.3=he6710b0_1
- libgcc-ng==9.1.0=hdf63c60_0
- libstdcxx-ng==9.1.0=hdf63c60_0
- ncurses==6.2=he6710b0_1
- openssl==1.1.1g=h7b6447c_0
- pip==20.0.2=py37_3
- pycosat==0.6.3=py37h7b6447c_0
- pycparser==2.20=py_0
- pyopenssl==19.1.0=py37_0
- pysocks==1.7.1=py37_0
- python==3.7.7=hcff3b4d_5
- readline==8.0=h7b6447c_0
- requests==2.23.0=py37_0
- ruamel_yaml==0.15.87=py37h7b6447c_0
- setuptools==46.4.0=py37_0
- six==1.14.0=py37_0
- sqlite==3.31.1=h62c20be_1
- tk==8.6.8=hbc83047_0
- tqdm==4.46.0=py_0
- urllib3==1.25.8=py37_0
- wheel==0.34.2=py37_0
- xz==5.2.5=h7b6447c_0
- yaml==0.1.7=had09818_2
- zlib==1.2.11=h7b6447c_3
The following NEW packages will be INSTALLED:
_libgcc_mutex pkgs/main/linux-64::_libgcc_mutex-0.1-main
ca-certificates pkgs/main/linux-64::ca-certificates-2020.1.1-0
certifi pkgs/main/linux-64::certifi-2020.4.5.1-py37_0
cffi pkgs/main/linux-64::cffi-1.14.0-py37he30daa8_1
chardet pkgs/main/linux-64::chardet-3.0.4-py37_1003
conda pkgs/main/linux-64::conda-4.8.3-py37_0
conda-package-han~ pkgs/main/linux-64::conda-package-handling-1.6.1-py37h7b6447c_0
cryptography pkgs/main/linux-64::cryptography-2.9.2-py37h1ba5d50_0
idna pkgs/main/noarch::idna-2.9-py_1
ld_impl_linux-64 pkgs/main/linux-64::ld_impl_linux-64-2.33.1-h53a641e_7
libedit pkgs/main/linux-64::libedit-3.1.20181209-hc058e9b_0
libffi pkgs/main/linux-64::libffi-3.3-he6710b0_1
libgcc-ng pkgs/main/linux-64::libgcc-ng-9.1.0-hdf63c60_0
libstdcxx-ng pkgs/main/linux-64::libstdcxx-ng-9.1.0-hdf63c60_0
ncurses pkgs/main/linux-64::ncurses-6.2-he6710b0_1
openssl pkgs/main/linux-64::openssl-1.1.1g-h7b6447c_0
pip pkgs/main/linux-64::pip-20.0.2-py37_3
pycosat pkgs/main/linux-64::pycosat-0.6.3-py37h7b6447c_0
pycparser pkgs/main/noarch::pycparser-2.20-py_0
pyopenssl pkgs/main/linux-64::pyopenssl-19.1.0-py37_0
pysocks pkgs/main/linux-64::pysocks-1.7.1-py37_0
python pkgs/main/linux-64::python-3.7.7-hcff3b4d_5
readline pkgs/main/linux-64::readline-8.0-h7b6447c_0
requests pkgs/main/linux-64::requests-2.23.0-py37_0
ruamel_yaml pkgs/main/linux-64::ruamel_yaml-0.15.87-py37h7b6447c_0
setuptools pkgs/main/linux-64::setuptools-46.4.0-py37_0
six pkgs/main/linux-64::six-1.14.0-py37_0
sqlite pkgs/main/linux-64::sqlite-3.31.1-h62c20be_1
tk pkgs/main/linux-64::tk-8.6.8-hbc83047_0
tqdm pkgs/main/noarch::tqdm-4.46.0-py_0
urllib3 pkgs/main/linux-64::urllib3-1.25.8-py37_0
wheel pkgs/main/linux-64::wheel-0.34.2-py37_0
xz pkgs/main/linux-64::xz-5.2.5-h7b6447c_0
yaml pkgs/main/linux-64::yaml-0.1.7-had09818_2
zlib pkgs/main/linux-64::zlib-1.2.11-h7b6447c_3
Preparing transaction: ...working... done
Executing transaction: ...working... done
installation finished.
$ source "$HOME/miniconda/etc/profile.d/conda.sh"
$ hash -r
$ conda config --set always_yes yes --set changeps1 no
$ conda update -q conda
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... done
## Package Plan ##
environment location: /root/miniconda
added / updated specs:
- conda
The following packages will be downloaded:
package | build
---------------------------|-----------------
brotlipy-0.7.0 |py37h7b6447c_1000 322 KB
ca-certificates-2020.6.24 | 0 125 KB
certifi-2020.6.20 | py37_0 156 KB
idna-2.10 | py_0 50 KB
libedit-3.1.20191231 | h14c3975_1 116 KB
libffi-3.3 | he6710b0_2 50 KB
pip-20.1.1 | py37_1 1.7 MB
pycparser-2.20 | py_2 94 KB
pyopenssl-19.1.0 | py_1 48 KB
pysocks-1.7.1 | py37_1 27 KB
requests-2.24.0 | py_0 56 KB
ruamel_yaml-0.15.87 | py37h7b6447c_1 245 KB
setuptools-49.2.0 | py37_0 743 KB
six-1.15.0 | py_0 13 KB
sqlite-3.32.3 | h62c20be_0 1.1 MB
tk-8.6.10 | hbc83047_0 3.0 MB
tqdm-4.47.0 | py_0 62 KB
urllib3-1.25.9 | py_0 103 KB
yaml-0.2.5 | h7b6447c_0 75 KB
------------------------------------------------------------
Total: 8.0 MB
The following NEW packages will be INSTALLED:
brotlipy pkgs/main/linux-64::brotlipy-0.7.0-py37h7b6447c_1000
The following packages will be UPDATED:
ca-certificates 2020.1.1-0 --> 2020.6.24-0
certifi 2020.4.5.1-py37_0 --> 2020.6.20-py37_0
idna 2.9-py_1 --> 2.10-py_0
libedit 3.1.20181209-hc058e9b_0 --> 3.1.20191231-h14c3975_1
libffi 3.3-he6710b0_1 --> 3.3-he6710b0_2
pip 20.0.2-py37_3 --> 20.1.1-py37_1
pycparser 2.20-py_0 --> 2.20-py_2
pyopenssl pkgs/main/linux-64::pyopenssl-19.1.0-~ --> pkgs/main/noarch::pyopenssl-19.1.0-py_1
pysocks 1.7.1-py37_0 --> 1.7.1-py37_1
requests pkgs/main/linux-64::requests-2.23.0-p~ --> pkgs/main/noarch::requests-2.24.0-py_0
ruamel_yaml 0.15.87-py37h7b6447c_0 --> 0.15.87-py37h7b6447c_1
setuptools 46.4.0-py37_0 --> 49.2.0-py37_0
six pkgs/main/linux-64::six-1.14.0-py37_0 --> pkgs/main/noarch::six-1.15.0-py_0
sqlite 3.31.1-h62c20be_1 --> 3.32.3-h62c20be_0
tk 8.6.8-hbc83047_0 --> 8.6.10-hbc83047_0
tqdm 4.46.0-py_0 --> 4.47.0-py_0
urllib3 pkgs/main/linux-64::urllib3-1.25.8-py~ --> pkgs/main/noarch::urllib3-1.25.9-py_0
yaml 0.1.7-had09818_2 --> 0.2.5-h7b6447c_0
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
$ conda info -a
active environment : None
shell level : 0
user config file : /root/.condarc
populated config files : /root/.condarc
conda version : 4.8.3
conda-build version : not installed
python version : 3.7.7.final.0
virtual packages : __glibc=2.24
base environment : /root/miniconda (writable)
channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/linux-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : /root/miniconda/pkgs
/root/.conda/pkgs
envs directories : /root/miniconda/envs
/root/.conda/envs
platform : linux-64
user-agent : conda/4.8.3 requests/2.24.0 CPython/3.7.7 Linux/4.19.78-coreos debian/9 glibc/2.24
UID:GID : 0:0
netrc file : None
offline mode : False
# conda environments:
#
base * /root/miniconda
sys.version: 3.7.7 (default, May 7 2020, 21:25:33)
...
sys.prefix: /root/miniconda
sys.executable: /root/miniconda/bin/python
conda location: /root/miniconda/lib/python3.7/site-packages/conda
conda-build: None
conda-env: /root/miniconda/bin/conda-env
user site dirs:
CIO_TEST: <not set>
CI_CONFIG_PATH: .gitlab-ci.yml
CI_PROJECT_PATH: omnieyes_general/OmniEyes
CONDA_EXE: /root/miniconda/bin/conda
CONDA_PYTHON_EXE: /root/miniconda/bin/python
CONDA_ROOT: /root/miniconda
CONDA_SHLVL: 0
PATH: /root/miniconda/bin:/root/miniconda/condabin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PYTHON_GET_PIP_SHA256: b3153ec0cf7b7bbf9556932aa37e4981c35dc2a2c501d70d91d2795aa532be79
PYTHON_GET_PIP_URL: https://github.com/pypa/get-pip/raw/eff16c878c7fd6b688b9b4c4267695cf1a0bf01b/get-pip.py
PYTHON_PIP_VERSION: 20.1.1
PYTHON_VERSION: 3.6.11
REQUESTS_CA_BUNDLE: <not set>
SSL_CERT_FILE: <not set>
$ conda env create -q -n omni-py36-tf112 -f conda/omni-py36-tf112.yml
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
$ conda activate omni-py36-tf112
$ free -h
total used free shared buff/cache available
Mem: 3.6G 485M 1.0G 189M 2.1G 2.7G
Swap: 2.0G 24M 2.0G
$ ulimit -s
8192
$ python -m unittest discover
/bin/bash: line 123: 2259 Segmentation fault (core dumped) python -m unittest discover
ERROR: Job failed: exit code 1Hope someone can give me some hint or it's kinda like I'm stuck here. Thanks in advance for any response.