Skip to content

fix(jemalloc): fix per-version src dir

Yoda BZH requested to merge Yoda-BZH/omnibus-gitlab:yodabzh-fix-jemalloc-per-version-dir into master
  • Please check this box if this contribution uses AI-generated content (including content generated by GitLab Duo features) as outlined in the GitLab DCO & CLA

What does this MR do?

fix(jemalloc): fix per-version src dir

While trying to build omnibus for exherbo, I had the following build error:

      [Builder: jemalloc] I | 2024-05-24T11:57:54+02:00 | Environment:
      [Builder: jemalloc] I | 2024-05-24T11:57:54+02:00 |   CFLAGS="-I/opt/gitlab/embedded/include -O3 -D_FORTIFY_SOURCE=2 -fstack-protector"
      [Builder: jemalloc] I | 2024-05-24T11:57:54+02:00 |   CPPFLAGS="-I/opt/gitlab/embedded/include -O3 -D_FORTIFY_SOURCE=2 -fstack-protector"
      [Builder: jemalloc] I | 2024-05-24T11:57:54+02:00 |   CXXFLAGS="-I/opt/gitlab/embedded/include -O3 -D_FORTIFY_SOURCE=2 -fstack-protector"
      [Builder: jemalloc] I | 2024-05-24T11:57:54+02:00 |   LDFLAGS="-Wl,-rpath,/opt/gitlab/embedded/lib -L/opt/gitlab/embedded/lib"
      [Builder: jemalloc] I | 2024-05-24T11:57:54+02:00 |   LD_RUN_PATH="/opt/gitlab/embedded/lib"
      [Builder: jemalloc] I | 2024-05-24T11:57:54+02:00 |   OMNIBUS_INSTALL_DIR="/opt/gitlab"
      [Builder: jemalloc] I | 2024-05-24T11:57:54+02:00 |   PATH="/opt/gitlab/bin:/opt/gitlab/embedded/bin:$HOME/$GEMDIR/gems/ruby/3.2/bin:$HOME/bin:/usr/local/bin:/usr/host/bin:/opt/bin"
      [Builder: jemalloc] I | 2024-05-24T11:57:54+02:00 |   PKG_CONFIG_PATH="/opt/gitlab/embedded/lib/pkgconfig"
      [Builder: jemalloc] I | 2024-05-24T11:57:54+02:00 | $ make -j 25 install
      [Builder: jemalloc] I | 2024-05-24T11:57:54+02:00 | Execute: `make -j 25 install': 38.1734s
      [Builder: jemalloc] I | 2024-05-24T11:57:54+02:00 | Build jemalloc: 45.8105s
The following shell command exited with status 2:

    $ CFLAGS=-I/opt/gitlab/embedded/include -O3 -D_FORTIFY_SOURCE=2 -fstack-protector CPPFLAGS=-I/opt/gitlab/embedded/include -O3 -D_FORTIFY_SOURCE=2 -fstack-protector CXXFLAGS=-I/opt/gitlab/embedded/include -O3 -D_FORTIFY_SOURCE=2 -fstack-protector LDFLAGS=-Wl,-rpath,/opt/gitlab/embedded/lib -L/opt/gitlab/embedded/lib LD_RUN_PATH=/opt/gitlab/embedded/lib OMNIBUS_INSTALL_DIR=/opt/gitlab PATH=/opt/gitlab/bin:/opt/gitlab/embedded/bin:$HOME/$GEMDIR/gems/ruby/3.2/bin:$HOME/bin:/usr/local/bin:/usr/host/bin:/opt/bin PKG_CONFIG_PATH=/opt/gitlab/embedded/lib/pkgconfig make -j 25 install

Output:

    /usr/host/bin/ginstall -c -d /opt/gitlab/embedded/bin
/usr/host/bin/ginstall -c -d /opt/gitlab/embedded/include/jemalloc
/bin/sh include/jemalloc/internal/private_namespace.sh src/jemalloc.sym src/arena.sym src/background_thread.sym src/base.sym src/bin.sym src/bin_info.sym src/bitmap.sym src/buf_writer.sym src/cache_bin.sym src/ckh.sym src/counter.sym src/ctl.sym src/decay.sym src/div.sym src/ecache.sym src/edata.sym src/edata_cache.sym src/ehooks.sym src/emap.sym src/eset.sym src/exp_grow.sym src/extent.sym src/extent_dss.sym src/extent_mmap.sym src/fxp.sym src/san.sym src/san_bump.sym src/hook.sym src/hpa.sym src/hpa_hooks.sym src/hpdata.sym src/inspect.sym src/large.sym src/log.sym src/malloc_io.sym src/mutex.sym src/nstime.sym src/pa.sym src/pa_extra.sym src/pai.sym src/pac.sym src/pages.sym src/peak_event.sym src/prof.sym src/prof_data.sym src/prof_log.sym src/prof_recent.sym src/prof_stats.sym src/prof_sys.sym src/psset.sym src/rtree.sym src/safety_check.sym src/sc.sym src/sec.sym src/stats.sym src/sz.sym src/tcache.sym src/test_hooks.sym src/thread_event.sym src/ticker.sym src/tsd.sym src/witness.sym > include/jemalloc/internal/private_namespace.gen.h
/usr/host/bin/ginstall -c -d /opt/gitlab/embedded/lib/pkgconfig
/usr/host/bin/xsltproc -o doc/jemalloc.html doc/html.xsl doc/jemalloc.xml
/usr/host/bin/xsltproc -o doc/jemalloc.3 doc/manpages.xsl doc/jemalloc.xml
removed '/opt/gitlab/embedded/include/jemalloc/jemalloc.h'
'include/jemalloc/jemalloc.h' -> '/opt/gitlab/embedded/include/jemalloc/jemalloc.h'
removed '/opt/gitlab/embedded/bin/jemalloc-config'
'bin/jemalloc-config' -> '/opt/gitlab/embedded/bin/jemalloc-config'
removed '/opt/gitlab/embedded/bin/jemalloc.sh'
'bin/jemalloc.sh' -> '/opt/gitlab/embedded/bin/jemalloc.sh'
removed '/opt/gitlab/embedded/lib/pkgconfig/jemalloc.pc'
'jemalloc.pc' -> '/opt/gitlab/embedded/lib/pkgconfig/jemalloc.pc'
removed '/opt/gitlab/embedded/bin/jeprof'
'bin/jeprof' -> '/opt/gitlab/embedded/bin/jeprof'

Error:

    doc/html.xsl:3: doc/manpages.xsl:3: parser parser error : error : Unescaped '<' not allowed in attributes values
Unescaped '<' not allowed in attributes values
  <xsl:import href="/var/cache/omnibus/src/jemalloc/jemalloc-#<Gitlab::Version:0
  <xsl:import href="/var/cache/omnibus/src/jemalloc/jemalloc-#<Gitlab::Version:0
                                                              ^
                                                              ^
doc/manpages.xsl:3: parser doc/html.xsl:3: error : parser attributes construct error
error : attributes construct error
  <xsl:import href="/var/cache/omnibus/src/jemalloc/jemalloc-#<Gitlab::Version:0
  <xsl:import href="/var/cache/omnibus/src/jemalloc/jemalloc-#<Gitlab::Version:0
                                                              ^
                                                              ^
doc/manpages.xsl:3: doc/html.xsl:3: parser parser error : error : Couldn't find end of Start Tag import line 3
Couldn't find end of Start Tag import line 3
  <xsl:import href="/var/cache/omnibus/src/jemalloc/jemalloc-#<Gitlab::Version:0
  <xsl:import href="/var/cache/omnibus/src/jemalloc/jemalloc-#<Gitlab::Version:0
                                                              ^
                                                              ^
doc/html.xsl:3: doc/manpages.xsl:3: namespace namespace error : error : Failed to parse QName 'Gitlab:'
Failed to parse QName 'Gitlab:'
  <xsl:import href="/var/cache/omnibus/src/jemalloc/jemalloc-#<Gitlab::Version:0
  <xsl:import href="/var/cache/omnibus/src/jemalloc/jemalloc-#<Gitlab::Version:0
                                                                      ^
                                                                      ^
doc/manpages.xsl:4: doc/html.xsl:5: parser parser error : error : Opening and ending tag mismatch: Gitlab::Version:0x00007efd743ceb50 line 3 and xsl:stylesheet
Opening and ending tag mismatch: Gitlab::Version:0x00007efd743ceb50 line 3 and xsl:stylesheet
</xsl:stylesheet>
                 ^
</xsl:stylesheet>
                 ^
doc/manpages.xsl:5: doc/html.xsl:6: parser parser error : error : Premature end of data in tag stylesheet line 1
Premature end of data in tag stylesheet line 1

^
^
cannot parse doc/html.xsl
cannot parse doc/manpages.xsl
make: *** [Makefile:383: doc/jemalloc.html] Error 4
make: *** Waiting for unfinished jobs....
make: *** [Makefile:393: doc/jemalloc.3] Error 4

$HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/util.rb:139:in `rescue in shellout!'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/util.rb:134:in `shellout!'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/builder.rb:911:in `shellout!'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/builder.rb:85:in `block in command'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/builder.rb:1101:in `instance_eval'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/builder.rb:1101:in `run'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/builder.rb:929:in `block (3 levels) in execute'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/builder.rb:951:in `with_retries'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/builder.rb:928:in `block (2 levels) in execute'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/instrumentation.rb:23:in `measure'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/builder.rb:927:in `block in execute'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/builder.rb:1002:in `with_clean_env'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/builder.rb:926:in `execute'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/builder.rb:818:in `block (2 levels) in build'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/builder.rb:817:in `each'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/builder.rb:817:in `block in build'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/instrumentation.rb:23:in `measure'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/builder.rb:816:in `build'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/software.rb:1371:in `execute_build'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/software.rb:1254:in `build_me'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/project.rb:1214:in `block (2 levels) in build'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/project.rb:1213:in `each'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/project.rb:1213:in `block in build'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/licensing.rb:59:in `block in create_incrementally'
  <internal:kernel>:90:in `tap'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/licensing.rb:54:in `create_incrementally'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/project.rb:1212:in `build'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/cli.rb:95:in `build'
  $HOME/$GEMDIR/gems/ruby/3.2/gems/thor-1.2.1/lib/thor/command.rb:27:in `run'
  $HOME/$GEMDIR/gems/ruby/3.2/gems/thor-1.2.1/lib/thor/invocation.rb:127:in `invoke_command'
  $HOME/$GEMDIR/gems/ruby/3.2/gems/thor-1.2.1/lib/thor.rb:392:in `dispatch'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/cli/base.rb:33:in `dispatch'
  $HOME/$GEMDIR/gems/ruby/3.2/gems/thor-1.2.1/lib/thor/base.rb:485:in `start'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/lib/omnibus/cli.rb:42:in `execute!'
  $HOME/$GEMDIR/gems/ruby/3.2/bundler/gems/omnibus-f48d9cd3d9e2/bin/omnibus:16:in `<top (required)>'
  $HOME/$GEMDIR/gems/ruby/3.2/bin/omnibus:25:in `load'
  $HOME/$GEMDIR/gems/ruby/3.2/bin/omnibus:25:in `<main>'
rake aborted!
Build failed

the generated directories are:

ls /var/cache/omnibus/src/jemalloc/
'jemalloc-#<Gitlab::Version:0x00007efd743ceb50>'  'jemalloc-#<Gitlab::Version:0x00007f81cafb4638>'

I suppose version from Gitlab::Version.new is an object, containing multiple fields.

Using version.print returns the proper version string, usable to create a per-version directory name.

Related issues

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion.

Required

  • MR title and description are up to date, accurate, and descriptive.
  • MR targeting the appropriate branch.
  • Latest Merge Result pipeline is green.
  • When ready for review, MR is labeled "~workflow::ready for review" per the Distribution MR workflow.

For GitLab team members

If you don't have access to this, the reviewer should trigger these jobs for you during the review process.

  • The manual Trigger:ee-package jobs have a green pipeline running against latest commit.
  • If config/software or config/patches directories are changed, make sure the build-package-on-all-os job within the Trigger:ee-package downstream pipeline succeeded.
  • If you are changing anything SSL related, then the Trigger:package:fips manual job within the Trigger:ee-package downstream pipeline must succeed.
  • If CI configuration is changed, the branch must be pushed to dev.gitlab.org to confirm regular branch builds aren't broken.

Expected (please provide an explanation if not completing)

  • Test plan indicating conditions for success has been posted and passes.
  • Documentation created/updated.
  • Tests added.
  • Integration tests added to GitLab QA.
  • Equivalent MR/issue for the GitLab Chart opened.
  • Validate potential values for new configuration settings. Formats such as integer 10, duration 10s, URI scheme://user:passwd@host:port may require quotation or other special handling when rendered in a template and written to a configuration file.
Edited by 🤖 GitLab Bot 🤖

Merge request reports