Add`status_message` to the package model

🌱 Context

When a package fails to upload to the GitLab Package Registry, we currently display only a generic message in the UI. The message does not give the user much insight into what went wrong and makes it difficult for them to know how to resolve the issue.

As part of the epic, &7137, we are hoping to change that. The goal is to surface helpful error messages in the UI so that when a user is troubleshooting a specific package, they can remediate the issue faster.

What does this MR do and why?

• Adds new status_message column to the package model
• Updates the extraction workers to add the exception message to the package.status_message column, for Debian, Helm, Nuget, and Rubygems.
  • The exception message is added as-is for common errors we have anticipated in the code.
  • For unanticipated errors, we use Unexpected error: <Exception class name>. The full exception message is usually cryptic and not very useful to the end-user for these cases.

Scenarios and Error Messages

Package format	Scenario	Exception	Error message
Debian	Debian package has no metadata	ArgumentError	package file without Debian metadata
Debian	Package file distribution_name is blank	ArgumentError	missing distribution name
Debian	Package file component_name is blank	ArgumentError	missing component name
Debian	package file is not deb, ddeb or udeb	ArgumentError	invalid package file type
Debian	package exists in another distribution	ArgumentError	Debian package #{package_name} #{package_version} exists in distribution in #{other_distribution_codename}
Debian	Changes file distribution_name is blank	ArgumentError	unwanted distribution name
Debian	Changes file component_name is blank	ArgumentError	unwanted component name
Debian	Changes file metadata has blank Source field	ArgumentError	missing Source field
Debian	Changes file metadata has blank Version field	ArgumentError	missing Version field
Debian	Changes file metadata has blank Distribution field	ArgumentError	missing Distribution field
Debian	All other errors	StandardError	Unexpected error: <exception class>
Helm	YAML has invalid values	ActiveRecord::Invalid	Validation failed
Helm	Empty package file	::Packages::Helm::ExtractFileMetadataService::ExtractionError	Chart.yaml not found within a directory
Helm	Invalid Chart.yaml	::Packages::Helm::ExtractFileMetadataService::ExtractionError	Error while parsing Chart.yaml
Helm	All other errors	StandardError	Unexpected error: <exception class>
Nuget	.nupkg file is not a valid Zip file	::Packages::Nuget::UpdatePackageFromMetadataService::ZipError	Could not open the .nupkg file
Nuget	.nupkg file has invalid metadata	::Packages::Nuget::UpdatePackageFromMetadataService::InvalidMetadataError	package name, version, authors and/or description not found in metadata
Nuget	All other errors	StandardError	Unexpected error: <exception class>
Rubygems	File I/O error while opening the gem file	::Packages::Rubygems::ProcessGemService::ExtractionError	Unable to read gem file
Rubygems	A metadata field exceeds the text limit	ActiveRecord::StatementInvalid	Unexpected error: ActiveRecord::StatementInvalid [1]
Rubygems	All other errors	StandardError	Unexpected error: <exception class>

[1] We can improve this to Validation failed: <metadata field name> is too long (maximum is <maxlength> characters) by adding ActiveRecord validations on the ::Packages::Rubygems::Metadatum model

Screenshots or screen recordings

None. This is a ~backend-only change.

How to set up and validate locally

1. Build a Rubygem bundle gem hello

2. Go into the generated hello directory and edit hello.gemspec. Add values to the TODO fields.

Example hello.gemspec file:

# frozen_string_literal: true

require_relative "lib/hello/version"

Gem::Specification.new do |spec|
  spec.name = "hello"
  spec.version = Hello::VERSION
  spec.authors = ["Radamanthus Batnag"]
  spec.email = ["rbatnag@gitlab.com"]

  spec.summary = "Real gem summary goes here"
  spec.description = "Real gem description goes here"
  spec.homepage = "http://hello.com"
  spec.required_ruby_version = ">= 2.6.0"

  spec.metadata["allowed_push_host"] = "http://gdk.test:3000"

  spec.metadata["homepage_uri"] = spec.homepage
  spec.metadata["source_code_uri"] = "http://gdk.test:3000"
  spec.metadata["changelog_uri"] = "http://gdk.test:3000"

  # Specify which files should be added to the gem when it is released.
  # The `git ls-files -z` loads the files in the RubyGem that have been added into git.
  spec.files = Dir.chdir(__dir__) do
    `git ls-files -z`.split("\x0").reject do |f|
      (f == __FILE__) || f.match(%r{\A(?:(?:bin|test|spec|features)/|\.(?:git|circleci)|appveyor)})
    end
  end
  spec.bindir = "exe"
  spec.executables = spec.files.grep(%r{\Aexe/}) { |f| File.basename(f) }
  spec.require_paths = ["lib"]

  # Uncomment to register a new dependency of your gem
  # spec.add_dependency "example-gem", "~> 1.0"

  # For more information and examples about making a new gem, check out our
  # guide at: https://bundler.io/guides/creating_gem.html
end

3. Enable the Rubygems feature from the Rails console Feature.enable(:rubygem_packages)

4. Setup ~/.gem/credentials as described here

5. Upload to gem to an existing project you have access to:

gem push hello-0.0.1.gem --host http://gdk.test:3000/api/v4/projects/<project_id>/packages/rubygems

6. The gem will be uploaded:

Pushing gem to http://gdk.test:3000/api/v4/projects/7/packages/rubygems...
{"message":"201 Created"}

7. But will encounter errors during processing. Check the errors in the Rails console.

::Packages::Package.last

There should be a value for the status_message column. Example output:

 id: 43,
 project_id: 7,
 created_at: Mon, 05 Jun 2023 15:14:12.122454000 UTC +00:00,
 updated_at: Mon, 05 Jun 2023 15:14:12.122454000 UTC +00:00,
 name: "Gem.Temporary.Package",
 version: "0.0.0-d19f1704-a188-4e03-8415-8df68d4ecc0a",
 package_type: "rubygems",
 creator_id: 1,
 status: "error",
 last_downloaded_at: nil,
 status_message: "Empty list of attributes passed">

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.

Migration Log

rad@mjolnirv3 ~/g/gitlab (364053-add-status_message-to-the-package-model)> be rails db:migrate:redo:main STEP=2
main: == [advisory_lock_connection] object_id: 227580, pg_backend_pid: 30994
main: == 20230601153401 AddTextLimitToPackagesStatusMessage: reverting ==============
main: -- transaction_open?()
main:    -> 0.0000s
main: -- transaction_open?()
main:    -> 0.0000s
main: -- execute("            ALTER TABLE packages_packages\n            DROP CONSTRAINT IF EXISTS check_d6301aedeb\n")
main:    -> 0.0009s
main: == 20230601153401 AddTextLimitToPackagesStatusMessage: reverted (0.0188s) =====

main: == 20230601090722 AddStatusMessageToPackages: reverting =======================
main: -- remove_column(:packages_packages, :status_message, :text)
main:    -> 0.0006s
main: == 20230601090722 AddStatusMessageToPackages: reverted (0.0053s) ==============

main: == [advisory_lock_connection] object_id: 227580, pg_backend_pid: 30994
main: == [advisory_lock_connection] object_id: 228960, pg_backend_pid: 31276
main: == 20230601090722 AddStatusMessageToPackages: migrating =======================
main: -- add_column(:packages_packages, :status_message, :text)
main:    -> 0.0021s
main: == 20230601090722 AddStatusMessageToPackages: migrated (0.0080s) ==============

main: == 20230601153401 AddTextLimitToPackagesStatusMessage: migrating ==============
main: -- transaction_open?()
main:    -> 0.0000s
main: -- transaction_open?()
main:    -> 0.0000s
main: -- execute("ALTER TABLE packages_packages\nADD CONSTRAINT check_d6301aedeb\nCHECK ( char_length(status_message) <= 255 )\nNOT VALID;\n")
main:    -> 0.0011s
main: == 20230601153401 AddTextLimitToPackagesStatusMessage: migrated (0.7366s) =====

main: == [advisory_lock_connection] object_id: 228960, pg_backend_pid: 31276

Related to #364053 (closed)