Unable to install cmake from apt.kitware.com
Thanks for maintaining these images! My name is James, I'm one of the maintainers of LightGBM
. We use some of the images from this repo in continuous integration jobs for CUDA-based builds of LightGBM. Those jobs started breaking a few days ago, and I think I've narrowed down why.
Reproducible Example
Run a shell in a container using one of the nvidia/cuda:{version}-devel
images.
docker run -it nvcr.io/nvidia/cuda:9.0-devel /bin/bash
Follow cmake
's documentation for installing on Ubuntu, by running the following...
apt-get update
apt-get install -y \
apt-transport-https wget
wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null \
| gpg --dearmor - \
| tee /usr/share/keyrings/kitware-archive-keyring.gpg >/dev/null
# returns non-0 code
echo $?
This step results in a non-0 exit code and the following message.
gpg: no valid OpenPGP data found
Running the same steps in a plain ubuntu:16.04
image, no such issue is encountered.
docker run -it ubuntu:16.04 /bin/bash
apt-get update
apt-get install -y \
apt-transport-https wget
wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null \
| gpg --dearmor - \
| tee /usr/share/keyrings/kitware-archive-keyring.gpg >/dev/null
# returns 0
echo $?
Follow the rest of Kitware's suggestions at https://apt.kitware.com/ to install cmake
results in the installation.
echo 'deb [signed-by=/usr/share/keyrings/kitware-archive-keyring.gpg] https://apt.kitware.com/ubuntu/ xenial main' \
| tee /etc/apt/sources.list.d/kitware.list >/dev/null
apt-get update
apt-get install -y cmake
cmake --version
In nvidia/cuda
images, the apt-get update
step here results in the following error.
Err:13 https://apt.kitware.com/ubuntu xenial/main amd64 Packages server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none Ign:14 https://apt.kitware.com/ubuntu xenial/main all Packages Reading package lists... Done W: The repository 'https://apt.kitware.com/ubuntu xenial Release' does not have a Release file. N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use. N: See apt-secure(8) manpage for repository creation and user configuration details. E: Failed to fetch https://apt.kitware.com/ubuntu/dists/xenial/main/binary-amd64/Packages server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none E: Some index files failed to download. They have been ignored, or old ones used instead.
In containers using nvidia/cuda
images, a very old cmake
version (3.5.1, from March 2016) is installed. In a container using the ubuntu:16.04
image, a much newer version (3.20.5, from June 2021) is installed.
This reproducible example uses nvcr.io/nvidia/cuda:9.0-devel
, but I see the same issues using nvcr.io/nvidia/cuda:10.0-devel-ubuntu18.04
and nvcr.io/nvidia/cuda:11.4.2-devel-ubuntu20.04
.
Notes
See https://github.com/microsoft/LightGBM/issues/4646 for more investigation.
Given the timing of this issue (CI jobs started failing around October 1st), I suspect that it's related to the expiration of Let's Encrypt's root certificate on September 30, 2021: https://scotthelme.co.uk/lets-encrypt-old-root-expiration/. But I don't know enough about TLS to investigate this more deeply.
Thanks very much for your time and consideration.