gitlab-exporter /metrics endpoint fails on UBI
Summary
Calling the /metrics endpoint on the UBI version of gitlab-export fails with the error:
LoadError: libpq.so.5: cannot open shared object file: No such file or directory - /usr/lib64/ruby/gems/2.7.0/gems/pg-1.2.3/lib/pg_ext.so
Steps to reproduce
- Validate that standard container /metrics endpoint works properly: registry.gitlab.com/gitlab-org/build/cng/gitlab-exporter:11.14.0
git@gitlab-toolbox-5b86d9bfdb-q65p6:/$ curl gitlab-gitlab-exporter:9168/metrics
ruby_gc_stat_count 28163
ruby_gc_stat_heap_allocated_pages 376
ruby_gc_stat_heap_sorted_length 526
...
- Switch to UBI container
kubectl set image pod/gitlab-gitlab-exporter-f5679456-csllc gitlab-exporter=registry.gitlab.com/gitlab-org/build/cng/gitlab-exporter:11.14.0-ubi8 -n gitlab
- Call /metrics endpoint on UBI container:
git@gitlab-toolbox-5b86d9bfdb-q65p6:/$ curl gitlab-gitlab-exporter:9168/metrics
LoadError: libpq.so.5: cannot open shared object file: No such file or directory - /usr/lib64/ruby/gems/2.7.0/gems/pg-1.2.3/lib/pg_ext.so
Expected behavior
/metrics endpoint works properly in both versions.
Relevant logs
Full error output from UBI container
git@gitlab-toolbox-5b86d9bfdb-q65p6:/$ curl gitlab-gitlab-exporter:9168/metrics
LoadError: libpq.so.5: cannot open shared object file: No such file or directory - /usr/lib64/ruby/gems/2.7.0/gems/pg-1.2.3/lib/pg_ext.so
/usr/lib64/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:85:in `require'
/usr/lib64/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:85:in `require'
/usr/lib64/ruby/gems/2.7.0/gems/pg-1.2.3/lib/pg.rb:5:in `<top (required)>'
/usr/lib64/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:85:in `require'
/usr/lib64/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:85:in `require'
/usr/lib64/ruby/gems/2.7.0/gems/gitlab-exporter-11.14.0/lib/gitlab_exporter/database/base.rb:1:in `<top (required)>'
/usr/lib64/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:85:in `require'
/usr/lib64/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:85:in `require'
/usr/lib64/ruby/gems/2.7.0/gems/gitlab-exporter-11.14.0/lib/gitlab_exporter/database/ci_builds.rb:5:in `<module:Database>'
/usr/lib64/ruby/gems/2.7.0/gems/gitlab-exporter-11.14.0/lib/gitlab_exporter/database/ci_builds.rb:3:in `<module:Exporter>'
/usr/lib64/ruby/gems/2.7.0/gems/gitlab-exporter-11.14.0/lib/gitlab_exporter/database/ci_builds.rb:2:in `<module:GitLab>'
/usr/lib64/ruby/gems/2.7.0/gems/gitlab-exporter-11.14.0/lib/gitlab_exporter/database/ci_builds.rb:1:in `<top (required)>'
/usr/lib64/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:85:in `require'
/usr/lib64/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:85:in `require'
/usr/lib64/ruby/gems/2.7.0/gems/gitlab-exporter-11.14.0/lib/gitlab_exporter/prober.rb:34:in `const_get'
/usr/lib64/ruby/gems/2.7.0/gems/gitlab-exporter-11.14.0/lib/gitlab_exporter/prober.rb:34:in `block (2 levels) in resolve_prober_classes'
/usr/lib64/ruby/gems/2.7.0/gems/gitlab-exporter-11.14.0/lib/gitlab_exporter/prober.rb:34:in `each'
/usr/lib64/ruby/gems/2.7.0/gems/gitlab-exporter-11.14.0/lib/gitlab_exporter/prober.rb:34:in `reduce'
/usr/lib64/ruby/gems/2.7.0/gems/gitlab-exporter-11.14.0/lib/gitlab_exporter/prober.rb:34:in `block in resolve_prober_classes'
/usr/lib64/ruby/gems/2.7.0/gems/gitlab-exporter-11.14.0/lib/gitlab_exporter/prober.rb:31:in `each'
/usr/lib64/ruby/gems/2.7.0/gems/gitlab-exporter-11.14.0/lib/gitlab_exporter/prober.rb:31:in `resolve_prober_classes'
/usr/lib64/ruby/gems/2.7.0/gems/gitlab-exporter-11.14.0/lib/gitlab_exporter/prober.rb:10:in `initialize'
/usr/lib64/ruby/gems/2.7.0/gems/gitlab-exporter-11.14.0/lib/gitlab_exporter/web_exporter.rb:92:in `new'
/usr/lib64/ruby/gems/2.7.0/gems/gitlab-exporter-11.14.0/lib/gitlab_exporter/web_exporter.rb:92:in `block (2 levels) in setup_probes'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1675:in `call'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1675:in `block in compile!'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1013:in `block (3 levels) in route!'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1032:in `route_eval'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1013:in `block (2 levels) in route!'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1061:in `block in process_route'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1059:in `catch'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1059:in `process_route'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1011:in `block in route!'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1008:in `each'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1008:in `route!'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1129:in `block in dispatch!'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1101:in `block in invoke'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1101:in `catch'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1101:in `invoke'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1124:in `dispatch!'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:939:in `block in call!'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1101:in `block in invoke'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1101:in `catch'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1101:in `invoke'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:939:in `call!'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:929:in `call'
/usr/lib64/ruby/gems/2.7.0/gems/gitlab-exporter-11.14.0/lib/gitlab_exporter/web_exporter.rb:47:in `call'
/usr/lib64/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/logger.rb:17:in `call'
/usr/lib64/ruby/gems/2.7.0/gems/gitlab-exporter-11.14.0/lib/gitlab_exporter/web_exporter.rb:21:in `call'
/usr/lib64/ruby/gems/2.7.0/gems/rack-protection-2.1.0/lib/rack/protection/xss_header.rb:18:in `call'
/usr/lib64/ruby/gems/2.7.0/gems/rack-protection-2.1.0/lib/rack/protection/path_traversal.rb:16:in `call'
/usr/lib64/ruby/gems/2.7.0/gems/rack-protection-2.1.0/lib/rack/protection/json_csrf.rb:26:in `call'
/usr/lib64/ruby/gems/2.7.0/gems/rack-protection-2.1.0/lib/rack/protection/base.rb:50:in `call'
/usr/lib64/ruby/gems/2.7.0/gems/rack-protection-2.1.0/lib/rack/protection/base.rb:50:in `call'
/usr/lib64/ruby/gems/2.7.0/gems/rack-protection-2.1.0/lib/rack/protection/frame_options.rb:31:in `call'
/usr/lib64/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/null_logger.rb:11:in `call'
/usr/lib64/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/head.rb:12:in `call'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/show_exceptions.rb:22:in `call'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:216:in `call'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1991:in `call'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1542:in `block in call'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1769:in `synchronize'
/usr/lib64/ruby/gems/2.7.0/gems/sinatra-2.1.0/lib/sinatra/base.rb:1542:in `call'
/usr/lib64/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/handler/webrick.rb:95:in `service'
/usr/lib64/ruby/2.7.0/webrick/httpserver.rb:140:in `service'
/usr/lib64/ruby/2.7.0/webrick/httpserver.rb:96:in `run'
/usr/lib64/ruby/2.7.0/webrick/server.rb:307:in `block in start_thread'