Container scans failing with runtime error

Summary

Running container scans with CS_IMAGE_SUFFIX: "-fips" results in runtime error when scanning some image (e.g.registry.gitlab.com/security-products/gemnasium:4-fips).

Steps to reproduce

  1. Configure the scan with below configuration
include:
  - template: Security/Container-Scanning.gitlab-ci.yml
container_scanning:
  variables:
    CS_IMAGE: "$IMAGE_ALIAS"
    CS_IMAGE_SUFFIX: "-fips"
    CS_IGNORE_STATUSES: "not_affected"
    SECURE_LOG_LEVEL: 'debug'
  parallel:
    matrix:
      - IMAGE_ALIAS:
        - registry.gitlab.com/gitlab-com/gl-infra/gitlab-dedicated/instrumentor/provision:main-fips #scan fails
        - registry.gitlab.com/security-products/gemnasium:4-fips #scan fails
        - registry.gitlab.com/security-products/gemnasium-python:4-fips #scan fails
        - registry.gitlab.com/gitlab-com/gl-infra/gitlab-dedicated/instrumentor/qa:main-fips #scan fails
        - registry.gitlab.com/gitlab-org/build/cng/gitlab-base:master-fips #scan run without error
  1. Except container_scanning: [registry.gitlab.com/gitlab-org/build/cng/gitlab-base:master-fips] job all the scan jobs fails with runtime error invalid byte sequence in US-ASCII

This issue does not seems to occur when CS_IMAGE_SUFFIX: "-fips" is not used.

Example Project

Scan with CS_IMAGE_SUFFIX: "-fips" which have failed scan jobs

  1. https://gitlab.com/ngeorge1/test-failing-scan/-/pipelines/1124674769

Scanning same images without CS_IMAGE_SUFFIX: "-fips" runs without errors.

  1. https://gitlab.com/ngeorge1/test-failing-scan/-/pipelines/1124677455

What is the current bug behavior?

Scan fails with runtime error.

What is the expected correct behavior?

Scan runs successfully.

Relevant logs and/or screenshots

This is a job log related to the error.

[ERROR] [2024-01-02 16:02:51 +0000] [container-scanning]  >  invalid byte sequence in US-ASCII
[ERROR] [2024-01-02 16:02:51 +0000] [container-scanning]  >  /home/gitlab/.local/share/gem/ruby/gems/gcs-6.6.1/lib/gcs/converter.rb:48:in `tr'
/home/gitlab/.local/share/gem/ruby/gems/gcs-6.6.1/lib/gcs/converter.rb:48:in `sanitized_source'
/home/gitlab/.local/share/gem/ruby/gems/gcs-6.6.1/lib/gcs/converter.rb:18:in `convert'
/home/gitlab/.local/share/gem/ruby/gems/gcs-6.6.1/lib/gcs/plugin/container_scan.rb:10:in `convert'
/home/gitlab/.local/share/gem/ruby/gems/gcs-6.6.1/lib/gcs/scan.rb:26:in `scan_image'
/home/gitlab/.local/share/gem/ruby/gems/gcs-6.6.1/lib/gcs/cli.rb:16:in `block in scan'
/home/gitlab/.local/share/gem/ruby/gems/gcs-6.6.1/lib/gcs/cli.rb:15:in `map'
/home/gitlab/.local/share/gem/ruby/gems/gcs-6.6.1/lib/gcs/cli.rb:15:in `scan'
/home/gitlab/.local/share/gem/ruby/gems/thor-1.3.0/lib/thor/command.rb:28:in `run'
/home/gitlab/.local/share/gem/ruby/gems/thor-1.3.0/lib/thor/invocation.rb:127:in `invoke_command'
/home/gitlab/.local/share/gem/ruby/gems/thor-1.3.0/lib/thor.rb:527:in `dispatch'
/home/gitlab/.local/share/gem/ruby/gems/thor-1.3.0/lib/thor/base.rb:584:in `start'
/home/gitlab/.local/share/gem/ruby/gems/gcs-6.6.1/exe/gtcs:14:in `<top (required)>'
/usr/bin/gtcs:25:in `load'
/usr/bin/gtcs:25:in `<main>'

Output of checks

This bug happens on GitLab.com

Implementation Plan

New proposal suggested during review:

  • set the proper utf8 locale when building the docker image
See original proposal 1. Pass an open option to specify the `encoding` when reading the trivy output file: 
       if status&.success? && File.exist?(OUTPUT_FILE)
-        scanner_output = File.read(OUTPUT_FILE)
+        # we need to force the encoding to UTF-8 because the fips Container Scanning image
+        # is incorrectly setting this to US-ASCII, which causes errors when reading UTF-8 chars
+        scanner_output = File.read(OUTPUT_FILE, :encoding => "UTF-8")
  1. Add a new integration test for the above behaviour, to prevent regressions.
Edited by Olivier Gonzalez