[#850] Shrink Docker images, avoid using pkg-config, avoid full libicu-dev,...

Sam Habielrequested to merge
shabiel/YDB:ydb850-redo into master

[#850 (closed)] Shrink Docker images, avoid using pkg-config, avoid full libicu-dev, avoid installing locales

The main Ubuntu docker image now is shrunk from 443MB to 224MB.

Rocky Linux image is now down from 571MB to 364MB.

Debian docker image left alone, as it contains the language bindings and needs a lot more analysis to actually decided how to shrink it down. It stands at about 1.5GB now.

Research contributed by @woji showed us the most of the space was taken by ICU libraries, pkg-config files, vim, locale definitions, and TLS certificate information.

This was accomplished as follows:

  • Avoid using pkg-config to find out what the ICU version is, and avoid using non-versioned ICU libraries provided in libicu-devel/ libicu-dev packages. This necessitated changing ydbenv.mpt and ydbinstall.sh to use ldconfig in order to find the correct version of the files for libicu.
  • Don't install vim. Use nano instead. This is more user friendly in any case.
  • Use Docker Buildkit to build the docker images. This is done to take advantage of a feature of mounting file systems from one image to another image, avoiding creating a new layer using COPY. See this blog post: https://rabbithole.wwwdotorg.org/2021/03/02/1-avoiding-docker-add-copy-layers.html. This necessitated adding a variable to .gitlab-ci.yml to enable this feature.
  • Remove locales package, as the Ubuntu image already has the C.UTF-8 locale enabled by default, so we actually were doing this step for absolutely no benefit. For Rocky Linux, don't install glibc-langpack-en. Rely on C.UTF-8 locale.
  • wget is required by ydbinstall, but not needed at runtime. It's the one that brings the TLS certificates along. Install it only before ydbinstall, but then remove it after and run apt autoclean to remove the rest.
  • Rather than libicu-dev, install the smaller libicu66, which is strictly only the runtime components. It's still huge (45MB by my estimation). Similar change for Rocky: only install libicu instead of libicu-devel
  • Remove /var/lib/apt/lists/* or yum stuff, as recommended by Docker documentation (https://docs.docker.com/develop/develop-images/dockerfile_best-practices/).
  • Reduce the docker image layers. This means less layers to download and unzip.
  • For Rocky Linux, make a YottaDB Release build, just like the Ubuntu Image. A RelWithDbgInfo build is almost twice the size (65MB -> 130MB).

More detail on ydbinstall.sh changes, as they are significant:

  • Add ldconfig to the list of required utilities
  • Create a new variable $ydb_found_or_requested_icu_version, independent of the user supplied $ydb_icu_version. This variable actually holds the actual version of the libicu libraries, or if a user supplies --utf8 nn, the value nn is used instead. This variable is used as well for plugin installs.

Testing:

  • Manually tested
  • sudo YDBTest suite passes
  • Docker image tested with this program created using nano editor:
YDB>zp ^foo
foo
        set x="こんにちは世界"
        write $ZCHSET,!
        write $length(x),!
        write x,!
        quit

YDB>d ^foo
UTF-8
7
こんにちは世界

Merge request reports