Skip to content

Force ffi gem to use Ruby platform gem

Stan Hu requested to merge sh-ffi-force-ruby-platform into master

What does this MR do?

ffi v1.17.0 started shipping precompiled native gems for x86 and aarch64 Linux. However, these precompiled gems don't include the Omnibus library path (/opt/gitlab/embedded/lib) as part of the default LD_LIBRARY_PATH, so libraries such as libarchive can't be loaded from Omnibus.

Bundler allows us to ignore the precompiled gems by using force_ruby_platform in the Gemfile, so use this feature to effectively restore the behavior in previous ffi gem versions.

Related issues

Relates to #8590 (closed)

Testing

$ /opt/gitlab/embedded/bin/gem list ffi

*** LOCAL GEMS ***

ffi (1.17.0, 1.15.5)
ffi-compiler (1.0.1)
ffi-libarchive (1.1.14)
ffi-yajl (2.6.0)
llhttp-ffi (0.4.0)
public_suffix (5.0.5, 5.0.0)

Previously:

$ /opt/gitlab/embedded/bin/gem list ffi

*** LOCAL GEMS ***

ffi (1.17.0 x86_64-linux-gnu, 1.15.5)
ffi-compiler (1.0.1)
ffi-libarchive (1.1.14)
ffi-yajl (2.6.0)
llhttp-ffi (0.4.0)
public_suffix (5.0.5, 5.0.0)

libarchive test

Now:

$ /opt/gitlab/embedded/bin/irb
irb(main):001:0> require 'ffi-libarchive'
=> true

On systems that do not have libarchive installed:

$ /opt/gitlab/embedded/bin/irb
irb(main):001:0> require 'ffi-libarchive'
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/ffi-1.17.0-x86_64-linux-gnu/lib/ffi/dynamic_library.rb:94:in `load_library': Could not open library 'libarchive.so.13': libarchive.so.13: cannot open shared object file: No such file or directory. (LoadError)
Could not open library 'libarchive.13': libarchive.13: cannot open shared object file: No such file or directory.
Could not open library 'libarchive.13.so': libarchive.13.so: cannot open shared object file: No such file or directory.
Could not open library 'libarchive-13': libarchive-13: cannot open shared object file: No such file or directory.
Could not open library 'libarchive-13.so': libarchive-13.so: cannot open shared object file: No such file or directory.
Could not open library 'libarchive.so': libarchive.so: cannot open shared object file: No such file or directory.
Could not open library 'libarchive': libarchive: cannot open shared object file: No such file or directory.
Could not open library 'libarchive.so': libarchive.so: cannot open shared object file: No such file or directory.
Could not open library 'archive': archive: cannot open shared object file: No such file or directory.
Could not open library 'libarchive.so': libarchive.so: cannot open shared object file: No such file or directory.
Searched in <system library path>
	from /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/ffi-1.17.0-x86_64-linux-gnu/lib/ffi/library.rb:95:in `block in ffi_lib'
	from /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/ffi-1.17.0-x86_64-linux-gnu/lib/ffi/library.rb:94:in `map'
	from /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/ffi-1.17.0-x86_64-linux-gnu/lib/ffi/library.rb:94:in `ffi_lib'
	from /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/ffi-libarchive-1.1.14/lib/ffi-libarchive/archive.rb:11:in `<module:C>'
	from /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/ffi-libarchive-1.1.14/lib/ffi-libarchive/archive.rb:4:in `<module:Archive>'
	from /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/ffi-libarchive-1.1.14/lib/ffi-libarchive/archive.rb:3:in `<top (required)>'
	from <internal:/opt/gitlab/embedded/lib/ruby/site_ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:136:in `require'
	from <internal:/opt/gitlab/embedded/lib/ruby/site_ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:136:in `require'
	from /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/ffi-libarchive-1.1.14/lib/ffi-libarchive.rb:56:in `<top (required)>'
	from <internal:/opt/gitlab/embedded/lib/ruby/site_ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:141:in `require'
	from <internal:/opt/gitlab/embedded/lib/ruby/site_ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:141:in `rescue in require'
	from <internal:/opt/gitlab/embedded/lib/ruby/site_ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:135:in `require'
	from (irb):1:in `<main>'
	from /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/irb-1.4.1/exe/irb:11:in `<top (required)>'
	from /opt/gitlab/embedded/bin/irb:25:in `load'
	from /opt/gitlab/embedded/bin/irb:25:in `<main>'
<internal:/opt/gitlab/embedded/lib/ruby/site_ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:136:in `require': cannot load such file -- ffi-libarchive (LoadError)
	from <internal:/opt/gitlab/embedded/lib/ruby/site_ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:136:in `require'
	from (irb):1:in `<main>'
	from /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/irb-1.4.1/exe/irb:11:in `<top (required)>'
	from /opt/gitlab/embedded/bin/irb:25:in `load'
	from /opt/gitlab/embedded/bin/irb:25:in `<main>'

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion.

Required

  • MR title and description are up to date, accurate, and descriptive.
  • MR targeting the appropriate branch.
  • Latest Merge Result pipeline is green.
  • When ready for review, MR is labeled "~workflow::ready for review" per the Distribution MR workflow.

For GitLab team members

If you don't have access to this, the reviewer should trigger these jobs for you during the review process.

  • The manual Trigger:ee-package jobs have a green pipeline running against latest commit.
  • If config/software or config/patches directories are changed, make sure the build-package-on-all-os job within the Trigger:ee-package downstream pipeline succeeded.
  • If you are changing anything SSL related, then the Trigger:package:fips manual job within the Trigger:ee-package downstream pipeline must succeed.
  • If CI configuration is changed, the branch must be pushed to dev.gitlab.org to confirm regular branch builds aren't broken.

Expected (please provide an explanation if not completing)

  • Test plan indicating conditions for success has been posted and passes.
  • Documentation created/updated.
  • Tests added.
  • Integration tests added to GitLab QA.
  • Equivalent MR/issue for the GitLab Chart opened.
  • Validate potential values for new configuration settings. Formats such as integer 10, duration 10s, URI scheme://user:passwd@host:port may require quotation or other special handling when rendered in a template and written to a configuration file.
Edited by Stan Hu

Merge request reports