Add`status_message` to the package model
🌱 Context
When a package fails to upload to the GitLab Package Registry, we currently display only a generic message in the UI. The message does not give the user much insight into what went wrong and makes it difficult for them to know how to resolve the issue.
As part of the epic, &7137, we are hoping to change that. The goal is to surface helpful error messages in the UI so that when a user is troubleshooting a specific package, they can remediate the issue faster.
What does this MR do and why?
- Adds new
status_message
column to the package model - Updates the extraction workers to add the exception message to the
package.status_message
column, for Debian, Helm, Nuget, and Rubygems.- The exception message is added as-is for common errors we have anticipated in the code.
- For unanticipated errors, we use
Unexpected error: <Exception class name>
. The full exception message is usually cryptic and not very useful to the end-user for these cases.
Scenarios and Error Messages
Package format | Scenario | Exception | Error message |
---|---|---|---|
Debian | Debian package has no metadata | ArgumentError | package file without Debian metadata |
Debian | Package file distribution_name is blank | ArgumentError | missing distribution name |
Debian | Package file component_name is blank | ArgumentError | missing component name |
Debian | package file is not deb, ddeb or udeb | ArgumentError | invalid package file type |
Debian | package exists in another distribution | ArgumentError | Debian package #{package_name} #{package_version} exists in distribution in #{other_distribution_codename} |
Debian | Changes file distribution_name is blank | ArgumentError | unwanted distribution name |
Debian | Changes file component_name is blank | ArgumentError | unwanted component name |
Debian | Changes file metadata has blank Source field | ArgumentError | missing Source field |
Debian | Changes file metadata has blank Version field | ArgumentError | missing Version field |
Debian | Changes file metadata has blank Distribution field | ArgumentError | missing Distribution field |
Debian | All other errors | StandardError | Unexpected error: <exception class>
|
Helm | YAML has invalid values | ActiveRecord::Invalid | Validation failed |
Helm | Empty package file | ::Packages::Helm::ExtractFileMetadataService::ExtractionError | Chart.yaml not found within a directory |
Helm | Invalid Chart.yaml | ::Packages::Helm::ExtractFileMetadataService::ExtractionError | Error while parsing Chart.yaml |
Helm | All other errors | StandardError | Unexpected error: <exception class>
|
Nuget | .nupkg file is not a valid Zip file | ::Packages::Nuget::UpdatePackageFromMetadataService::ZipError | Could not open the .nupkg file |
Nuget | .nupkg file has invalid metadata | ::Packages::Nuget::UpdatePackageFromMetadataService::InvalidMetadataError | package name, version, authors and/or description not found in metadata |
Nuget | All other errors | StandardError | Unexpected error: <exception class>
|
Rubygems | File I/O error while opening the gem file | ::Packages::Rubygems::ProcessGemService::ExtractionError | Unable to read gem file |
Rubygems | A metadata field exceeds the text limit | ActiveRecord::StatementInvalid | Unexpected error: ActiveRecord::StatementInvalid [1] |
Rubygems | All other errors | StandardError | Unexpected error: <exception class>
|
[1] We can improve this to Validation failed: <metadata field name> is too long (maximum is <maxlength> characters)
by adding ActiveRecord validations on the ::Packages::Rubygems::Metadatum
model
Screenshots or screen recordings
None. This is a ~backend-only change.
How to set up and validate locally
-
Build a Rubygem
bundle gem hello
-
Go into the generated
hello
directory and edithello.gemspec
. Add values to theTODO
fields.
Example hello.gemspec
file:
# frozen_string_literal: true
require_relative "lib/hello/version"
Gem::Specification.new do |spec|
spec.name = "hello"
spec.version = Hello::VERSION
spec.authors = ["Radamanthus Batnag"]
spec.email = ["rbatnag@gitlab.com"]
spec.summary = "Real gem summary goes here"
spec.description = "Real gem description goes here"
spec.homepage = "http://hello.com"
spec.required_ruby_version = ">= 2.6.0"
spec.metadata["allowed_push_host"] = "http://gdk.test:3000"
spec.metadata["homepage_uri"] = spec.homepage
spec.metadata["source_code_uri"] = "http://gdk.test:3000"
spec.metadata["changelog_uri"] = "http://gdk.test:3000"
# Specify which files should be added to the gem when it is released.
# The `git ls-files -z` loads the files in the RubyGem that have been added into git.
spec.files = Dir.chdir(__dir__) do
`git ls-files -z`.split("\x0").reject do |f|
(f == __FILE__) || f.match(%r{\A(?:(?:bin|test|spec|features)/|\.(?:git|circleci)|appveyor)})
end
end
spec.bindir = "exe"
spec.executables = spec.files.grep(%r{\Aexe/}) { |f| File.basename(f) }
spec.require_paths = ["lib"]
# Uncomment to register a new dependency of your gem
# spec.add_dependency "example-gem", "~> 1.0"
# For more information and examples about making a new gem, check out our
# guide at: https://bundler.io/guides/creating_gem.html
end
-
Enable the Rubygems feature from the Rails console
Feature.enable(:rubygem_packages)
-
Setup
~/.gem/credentials
as described here -
Upload to gem to an existing project you have access to:
gem push hello-0.0.1.gem --host http://gdk.test:3000/api/v4/projects/<project_id>/packages/rubygems
- The gem will be uploaded:
Pushing gem to http://gdk.test:3000/api/v4/projects/7/packages/rubygems...
{"message":"201 Created"}
- But will encounter errors during processing. Check the errors in the Rails console.
::Packages::Package.last
There should be a value for the status_message
column.
Example output:
id: 43,
project_id: 7,
created_at: Mon, 05 Jun 2023 15:14:12.122454000 UTC +00:00,
updated_at: Mon, 05 Jun 2023 15:14:12.122454000 UTC +00:00,
name: "Gem.Temporary.Package",
version: "0.0.0-d19f1704-a188-4e03-8415-8df68d4ecc0a",
package_type: "rubygems",
creator_id: 1,
status: "error",
last_downloaded_at: nil,
status_message: "Empty list of attributes passed">
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Migration Log
rad@mjolnirv3 ~/g/gitlab (364053-add-status_message-to-the-package-model)> be rails db:migrate:redo:main STEP=2
main: == [advisory_lock_connection] object_id: 227580, pg_backend_pid: 30994
main: == 20230601153401 AddTextLimitToPackagesStatusMessage: reverting ==============
main: -- transaction_open?()
main: -> 0.0000s
main: -- transaction_open?()
main: -> 0.0000s
main: -- execute(" ALTER TABLE packages_packages\n DROP CONSTRAINT IF EXISTS check_d6301aedeb\n")
main: -> 0.0009s
main: == 20230601153401 AddTextLimitToPackagesStatusMessage: reverted (0.0188s) =====
main: == 20230601090722 AddStatusMessageToPackages: reverting =======================
main: -- remove_column(:packages_packages, :status_message, :text)
main: -> 0.0006s
main: == 20230601090722 AddStatusMessageToPackages: reverted (0.0053s) ==============
main: == [advisory_lock_connection] object_id: 227580, pg_backend_pid: 30994
main: == [advisory_lock_connection] object_id: 228960, pg_backend_pid: 31276
main: == 20230601090722 AddStatusMessageToPackages: migrating =======================
main: -- add_column(:packages_packages, :status_message, :text)
main: -> 0.0021s
main: == 20230601090722 AddStatusMessageToPackages: migrated (0.0080s) ==============
main: == 20230601153401 AddTextLimitToPackagesStatusMessage: migrating ==============
main: -- transaction_open?()
main: -> 0.0000s
main: -- transaction_open?()
main: -> 0.0000s
main: -- execute("ALTER TABLE packages_packages\nADD CONSTRAINT check_d6301aedeb\nCHECK ( char_length(status_message) <= 255 )\nNOT VALID;\n")
main: -> 0.0011s
main: == 20230601153401 AddTextLimitToPackagesStatusMessage: migrated (0.7366s) =====
main: == [advisory_lock_connection] object_id: 228960, pg_backend_pid: 31276
Related to #364053 (closed)