Skip to content

Master pipeline broken because of `ruby-magic` build failures on several OSs

As seen from https://dev.gitlab.org/gitlab/omnibus-gitlab/-/pipelines/275805/failures

There are two variants of the failure

  1. Failing healthcheck, as seen from https://dev.gitlab.org/gitlab/omnibus-gitlab/-/jobs/15460151 and https://dev.gitlab.org/gitlab/omnibus-gitlab/-/jobs/15459972 
            [HealthCheck] I | 2023-03-14T03:47:02+00:00 | Running health on gitlab-ee
            [HealthCheck] E | 2023-03-14T03:47:34+00:00 | Failed!
            [HealthCheck] E | 2023-03-14T03:47:34+00:00 | The following libraries have unsafe or unmet dependencies:
    --> /opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/ruby-magic-0.5.5/lib/magic/magic.so
    --> /opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/ruby-magic-0.5.5/ext/magic/magic.so
    --> /opt/gitlab/embedded/lib/ruby/gems/3.0.0/extensions/aarch64-linux/3.0.0/ruby-magic-0.5.5/magic/magic.so
            [HealthCheck] E | 2023-03-14T03:47:34+00:00 | The following binaries have unsafe or unmet dependencies:
            [HealthCheck] E | 2023-03-14T03:47:34+00:00 | The following libraries cannot be guaranteed to be on target systems:
    --> /lib64/libzstd.so.1 (0x0000ffff92773000)
    --> /lib64/libzstd.so.1 (0x0000ffffb1f6f000)
    --> /lib64/libzstd.so.1 (0x0000ffff8f0ee000)
            [HealthCheck] E | 2023-03-14T03:47:34+00:00 | The precise failures were:
    --> /opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/ruby-magic-0.5.5/lib/magic/magic.so
    DEPENDS ON: libzstd.so.1
      COUNT: 1
      PROVIDED BY: /lib64/libzstd.so.1 (0x0000ffff92773000)
      FAILED BECAUSE: Unsafe dependency
    --> /opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/ruby-magic-0.5.5/ext/magic/magic.so
    DEPENDS ON: libzstd.so.1
      COUNT: 1
      PROVIDED BY: /lib64/libzstd.so.1 (0x0000ffffb1f6f000)
      FAILED BECAUSE: Unsafe dependency
    --> /opt/gitlab/embedded/lib/ruby/gems/3.0.0/extensions/aarch64-linux/3.0.0/ruby-magic-0.5.5/magic/magic.so
    DEPENDS ON: libzstd.so.1
      COUNT: 1
      PROVIDED BY: /lib64/libzstd.so.1 (0x0000ffff8f0ee000)
      FAILED BECAUSE: Unsafe dependency
            [HealthCheck] I | 2023-03-14T03:47:34+00:00 | Health check time: 32.0918s
  1. Failure to build, as seen from https://dev.gitlab.org/gitlab/omnibus-gitlab/-/jobs/15459934 and https://dev.gitlab.org/gitlab/omnibus-gitlab/-/jobs/15459971 
Installing ruby-magic 0.5.5 with native extensions
Error:
    [DEPRECATED] The `--without` flag is deprecated because it relies on being remembered across bundler invocations, which bundler will no longer do in future versions. Instead please use `bundle config set --local without 'development test'`, and stop using this flag
Don't run Bundler as root. Bundler can ask for sudo if it is needed, and
installing your bundle as root will break this application for all non-root
users on this machine.
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
current directory:
/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/ruby-magic-0.5.5/ext/magic
/opt/gitlab/embedded/bin/ruby -I /opt/gitlab/embedded/lib/ruby/3.0.0 -r
./siteconf20230314-13573-2bpbfn.rb extconf.rb
checking for whether -I/opt/gitlab/embedded/include is accepted as CFLAGS... yes
checking for whether -O3 is accepted as CFLAGS... yes
checking for whether -D_FORTIFY_SOURCE=2 is accepted as CFLAGS... yes
checking for whether -fstack-protector is accepted as CFLAGS... yes
checking for whether -I/opt/gitlab/embedded/include is accepted as CPPFLAGS...
yes
checking for whether -O3 is accepted as CPPFLAGS... yes
checking for whether -D_FORTIFY_SOURCE=2 is accepted as CPPFLAGS... yes
checking for whether -fstack-protector is accepted as CPPFLAGS... yes
checking for whether -Wl,-rpath,/opt/gitlab/embedded/lib is accepted as
LDFLAGS... yes
checking for whether -L/opt/gitlab/embedded/lib is accepted as LDFLAGS... yes
Building ruby-magic using packaged libraries.
Static linking is enabled.
Cross build is disabled.
Using mini_portile version 2.8.1
---------- IMPORTANT NOTICE ----------
Building Ruby Magic with a packaged version of libmagic-5.44.
Configuration options: --disable-silent-rules --disable-dependency-tracking
--enable-fsect-man5 --disable-shared --enable-static
CFLAGS\=-I/opt/gitlab/embedded/include\ -O3\ -D_FORTIFY_SOURCE\=2\
-fstack-protector\ -fPIC
The following patches are being applied:
  - 0001-Don-t-attempt-to-build-tests-documentation-and-Python-bindings.patch
Extracting file-5.44.tar.gz into tmp/x86_64-pc-linux-gnu/ports/libmagic/5.44...
OK
Running git apply with
/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/ruby-magic-0.5.5/patches/libmagic/0001-Don-t-attempt-to-build-tests-documentation-and-Python-bindings.patch...
OK
Running 'configure' for libmagic 5.44... OK
Running 'compile' for libmagic 5.44... ERROR, review
'/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/ruby-magic-0.5.5/ext/magic/tmp/x86_64-pc-linux-gnu/ports/libmagic/5.44/compile.log'
to see what happened. Last lines are:
========================================================================
/bin/sh ../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I.. 
-DMAGIC='"/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/ruby-magic-0.5.5/ports/x86_64-pc-linux-gnu/libmagic/5.44/share/misc/magic"'
-I/opt/gitlab/embedded/include -O3 -D_FORTIFY_SOURCE=2 -fstack-protector
-fvisibility=hidden -Wall -Wstrict-prototypes -Wmissing-prototypes
-Wpointer-arith        -Wmissing-declarations -Wredundant-decls -Wnested-externs
-Wsign-compare -Wreturn-type -Wswitch -Wshadow        -Wcast-qual
-Wwrite-strings -Wextra -Wunused-parameter -Wformat=2
-I/opt/gitlab/embedded/include -O3 -D_FORTIFY_SOURCE=2 -fstack-protector -fPIC
-c -o fsmagic.lo fsmagic.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I..
-DMAGIC=\"/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/ruby-magic-0.5.5/ports/x86_64-pc-linux-gnu/libmagic/5.44/share/misc/magic\"
-I/opt/gitlab/embedded/include -O3 -D_FORTIFY_SOURCE=2 -fstack-protector
-fvisibility=hidden -Wall -Wstrict-prototypes -Wmissing-prototypes
-Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs
-Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings
-Wextra -Wunused-parameter -Wformat=2 -I/opt/gitlab/embedded/include -O3
-D_FORTIFY_SOURCE=2 -fstack-protector -fPIC -c fsmagic.c  -fPIC -DPIC -o
fsmagic.o
/bin/sh ../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I.. 
-DMAGIC='"/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/ruby-magic-0.5.5/ports/x86_64-pc-linux-gnu/libmagic/5.44/share/misc/magic"'
-I/opt/gitlab/embedded/include -O3 -D_FORTIFY_SOURCE=2 -fstack-protector
-fvisibility=hidden -Wall -Wstrict-prototypes -Wmissing-prototypes
-Wpointer-arith        -Wmissing-declarations -Wredundant-decls -Wnested-externs
-Wsign-compare -Wreturn-type -Wswitch -Wshadow        -Wcast-qual
-Wwrite-strings -Wextra -Wunused-parameter -Wformat=2
-I/opt/gitlab/embedded/include -O3 -D_FORTIFY_SOURCE=2 -fstack-protector -fPIC
-c -o funcs.lo funcs.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I..
-DMAGIC=\"/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/ruby-magic-0.5.5/ports/x86_64-pc-linux-gnu/libmagic/5.44/share/misc/magic\"
-I/opt/gitlab/embedded/include -O3 -D_FORTIFY_SOURCE=2 -fstack-protector
-fvisibility=hidden -Wall -Wstrict-prototypes -Wmissing-prototypes
-Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs
-Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings
-Wextra -Wunused-parameter -Wformat=2 -I/opt/gitlab/embedded/include -O3
-D_FORTIFY_SOURCE=2 -fstack-protector -fPIC -c funcs.c  -fPIC -DPIC -o funcs.o
funcs.c: In function 'check_regex':
funcs.c:665:2: error: 'for' loop initial declarations are only allowed in C99
mode
  for (const char *p = pat; *p; p++) {
  ^
funcs.c:665:2: note: use option -std=c99 or -std=gnu99 to compile your code
Makefile:576: recipe for target 'funcs.lo' failed
make[3]: *** [funcs.lo] Error 1
make[3]: Leaving directory
'/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/ruby-magic-0.5.5/ext/magic/tmp/x86_64-pc-linux-gnu/ports/libmagic/5.44/file-5.44/src'
Makefile:381: recipe for target 'all' failed
make[2]: *** [all] Error 2
make[2]: Leaving directory
'/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/ruby-magic-0.5.5/ext/magic/tmp/x86_64-pc-linux-gnu/ports/libmagic/5.44/file-5.44/src'
Makefile:462: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory
'/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/ruby-magic-0.5.5/ext/magic/tmp/x86_64-pc-linux-gnu/ports/libmagic/5.44/file-5.44'
Makefile:371: recipe for target 'all' failed
make: *** [all] Error 2
========================================================================
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.
Provided configuration options:
	--with-opt-dir
	--with-opt-include
	--without-opt-include=${opt-dir}/include
	--with-opt-lib
	--without-opt-lib=${opt-dir}/lib
	--with-make-prog
	--without-make-prog
	--srcdir=.
	--curdir
	--ruby=/opt/gitlab/embedded/bin/$(RUBY_BASE_NAME)
	--help
	--clean
	--enable-system-libraries
	--disable-system-libraries
	--use-system-libraries
	--enable-static
	--disable-static
	--enable-cross-build
	--disable-cross-build
/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/mini_portile2-2.8.1/lib/mini_portile2/mini_portile.rb:460:in
`block in execute': Failed to complete compile task (RuntimeError)
from
/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/mini_portile2-2.8.1/lib/mini_portile2/mini_portile.rb:426:in
`chdir'
from
/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/mini_portile2-2.8.1/lib/mini_portile2/mini_portile.rb:426:in
`execute'
from
/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/mini_portile2-2.8.1/lib/mini_portile2/mini_portile.rb:144:in
`compile'
from
/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/mini_portile2-2.8.1/lib/mini_portile2/mini_portile.rb:187:in
`cook'
	from extconf.rb:161:in `block in process_recipe'
	from <internal:kernel>:90:in `tap'
	from extconf.rb:74:in `process_recipe'
	from extconf.rb:285:in `<main>'
To see why this extension failed to compile, please check the mkmf.log which can
be found here:
/opt/gitlab/embedded/lib/ruby/gems/3.0.0/extensions/x86_64-linux/3.0.0/ruby-magic-0.5.5/mkmf.log
extconf failed, exit code 1
Gem files will remain installed in
/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/ruby-magic-0.5.5 for inspection.
Results logged to
/opt/gitlab/embedded/lib/ruby/gems/3.0.0/extensions/x86_64-linux/3.0.0/ruby-magic-0.5.5/gem_make.out
  /opt/gitlab/embedded/lib/ruby/3.0.0/rubygems/ext/builder.rb:93:in `run'
/opt/gitlab/embedded/lib/ruby/3.0.0/rubygems/ext/ext_conf_builder.rb:47:in
`block in build'
  /opt/gitlab/embedded/lib/ruby/3.0.0/tempfile.rb:317:in `open'
/opt/gitlab/embedded/lib/ruby/3.0.0/rubygems/ext/ext_conf_builder.rb:26:in
`build'
/opt/gitlab/embedded/lib/ruby/3.0.0/rubygems/ext/builder.rb:159:in
`build_extension'
/opt/gitlab/embedded/lib/ruby/3.0.0/rubygems/ext/builder.rb:193:in `block in
build_extensions'
  /opt/gitlab/embedded/lib/ruby/3.0.0/rubygems/ext/builder.rb:190:in `each'
/opt/gitlab/embedded/lib/ruby/3.0.0/rubygems/ext/builder.rb:190:in
`build_extensions'
/opt/gitlab/embedded/lib/ruby/3.0.0/rubygems/installer.rb:837:in
`build_extensions'
/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/bundler-2.3.15/lib/bundler/rubygems_gem_installer.rb:71:in
`build_extensions'
/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/bundler-2.3.15/lib/bundler/rubygems_gem_installer.rb:28:in
`install'
/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/bundler-2.3.15/lib/bundler/source/rubygems.rb:204:in
`install'
/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/bundler-2.3.15/lib/bundler/installer/gem_installer.rb:54:in
`install'
/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/bundler-2.3.15/lib/bundler/installer/gem_installer.rb:16:in
`install_from_spec'
/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/bundler-2.3.15/lib/bundler/installer/parallel_installer.rb:186:in
`do_install'
/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/bundler-2.3.15/lib/bundler/installer/parallel_installer.rb:177:in
`block in worker_pool'
/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/bundler-2.3.15/lib/bundler/worker.rb:62:in
`apply_func'
/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/bundler-2.3.15/lib/bundler/worker.rb:57:in
`block in process_queue'
/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/bundler-2.3.15/lib/bundler/worker.rb:54:in
`loop'
/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/bundler-2.3.15/lib/bundler/worker.rb:54:in
`process_queue'
/opt/gitlab/embedded/lib/ruby/gems/3.0.0/gems/bundler-2.3.15/lib/bundler/worker.rb:91:in
`block (2 levels) in create_threads'
An error occurred while installing ruby-magic (0.5.5), and Bundler cannot
continue.
In Gemfile:
  ruby-magic