NoMethodError during GitLab upgrade from 18.1 to 18.2 when pushing to projects
Summary
During GitLab upgrades from version 18.1 to 18.2, the Project#csp_enabled? method throws a NoMethodError when attempting to access organization policy settings before post-deployment migrations have completed. This occurs because MR !196402 (merged) changed the method to rely on data in the organization_policy_setting table, but this table is only populated during post-deployment migrations that run at the end of the Zero-downtime upgrade process.
Steps to Reproduce
- Follow Zero-downtime upgrade process to upgrade GitLab from version 18.1 to 18.2. Do not run post-deployment migrations.
- Attempt to push code to any projects that existed before the upgrade started. This will trigger
Project#csp_enabled?method. - The method will attempt to delegate to
organization_policy_setting.csp_enabled?whenorganization_policy_settingis nil
Post upgrade:
gitlabhq_production=# select * from security_policy_settings;
id | csp_namespace_id | singleton | organization_id
----+------------------+-----------+-----------------
1 | | t |
(1 row)Current Behavior
-
NoMethodErroris thrown whencsp_enabled?is called on a nilorganization_policy_settingobject - Users cannot push code or access project features until post-deployment migrations complete
- Zero-downtime upgrades are broken due to this error
Expected Behavior
- The
csp_enabled?method should gracefully handle missing organization policy settings - GitLab should remain functional during the upgrade process before post-deployment migrations complete
Possible Fixes
Implemented Solution (merged in MR !203295 (merged)): Replace delegation with explicit method that uses safe navigation:
def csp_enabled?(group)
!!organization_policy_setting&.csp_enabled?(group)
endLogs and screenshots:
The following exception is generated:
"exception.backtrace": [
"ee/app/models/concerns/security/organization_policy_setting.rb:9:in `rescue in csp_enabled?'",
"ee/app/models/concerns/security/organization_policy_setting.rb:9:in `csp_enabled?'",
"ee/app/models/ee/project.rb:1542:in `security_orchestration_policy_configuration_parent_group_ids'",
"ee/app/models/ee/project.rb:1310:in `all_security_orchestration_policy_configurations'",
"ee/app/services/security/security_orchestration_policies/protected_branches_push_service.rb:24:in `applicable_active_policies'",
"ee/app/services/security/security_orchestration_policies/protected_branches_push_service.rb:13:in `rules'",
"ee/app/services/security/security_orchestration_policies/protected_branches_push_service.rb:7:in `execute'",
"ee/lib/ee/gitlab/checks/security/policy_check.rb:31:in `branch_name_affected_by_policy?'",
"ee/lib/ee/gitlab/checks/security/policy_check.rb:16:in `block in validate!'",
"lib/gitlab/checks/timed_logger.rb:27:in `log_timed'",
"ee/lib/ee/gitlab/checks/security/policy_check.rb:15:in `validate!'",
"lib/gitlab/checks/single_change_access.rb:58:in `ref_level_checks'",
"lib/gitlab/checks/single_change_access.rb:32:in `validate!'",
"lib/gitlab/checks/changes_access.rb:117:in `block in single_access_checks!'",
"lib/gitlab/checks/changes_access.rb:116:in `each'",
"lib/gitlab/checks/changes_access.rb:116:in `single_access_checks!'",
"lib/gitlab/checks/changes_access.rb:27:in `validate!'",
"lib/gitlab/git_access.rb:391:in `check_access!'",
"lib/gitlab/git_access.rb:378:in `check_change_access!'",
"ee/lib/ee/gitlab/git_access.rb:111:in `check_change_access!'",
"lib/gitlab/git_access.rb:359:in `check_push_access!'",
"lib/gitlab/git_access.rb:96:in `check'",
"ee/lib/ee/gitlab/git_access.rb:22:in `check'",
"lib/api/helpers/internal_helpers.rb:54:in `access_check!'",
"lib/api/helpers/internal_helpers.rb:35:in `block in access_check_result'",
"lib/gitlab/auth/current_user_mode.rb:45:in `bypass_session!'",
"lib/api/helpers/internal_helpers.rb:132:in `with_admin_mode_bypass!'",
"lib/api/helpers/internal_helpers.rb:34:in `access_check_result'",
"ee/lib/ee/api/internal/base.rb:90:in `access_check_result'",
"lib/api/internal/base.rb:88:in `check_allowed'",
"ee/lib/ee/api/internal/base.rb:22:in `block in check_allowed'",
"lib/gitlab/ip_address_state.rb:11:in `with'",
"ee/lib/ee/api/internal/base.rb:21:in `check_allowed'",
"lib/api/internal/base.rb:182:in `block (2 levels) in <class:Base>'",
"lib/gitlab/middleware/ip_address.rb:11:in `call'",
"lib/api/api_guard.rb:266:in `call'",
"config/initializers/action_dispatch_journey_router.rb:52:in `block in find_routes'",
"config/initializers/action_dispatch_journey_router.rb:25:in `map!'",
"config/initializers/action_dispatch_journey_router.rb:25:in `find_routes'",
"lib/gitlab/metrics/elasticsearch_rack_middleware.rb:16:in `call'",
"lib/gitlab/middleware/sidekiq_shard_awareness_validation.rb:20:in `block in call'",
"lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'",
"lib/gitlab/middleware/sidekiq_shard_awareness_validation.rb:20:in `call'",
"lib/gitlab/middleware/memory_report.rb:13:in `call'",
"lib/gitlab/middleware/speedscope.rb:13:in `call'",
"lib/gitlab/database/load_balancing/rack_middleware.rb:23:in `call'",
"lib/gitlab/middleware/rails_queue_duration.rb:33:in `call'",
"lib/gitlab/etag_caching/middleware.rb:21:in `call'",
"lib/gitlab/metrics/rack_middleware.rb:16:in `block in call'",
"lib/gitlab/metrics/web_transaction.rb:46:in `run'",
"lib/gitlab/metrics/rack_middleware.rb:16:in `call'",
"lib/gitlab/middleware/go.rb:21:in `call'",
"lib/gitlab/middleware/query_analyzer.rb:11:in `block in call'",
"lib/gitlab/database/query_analyzer.rb:83:in `within'",
"lib/gitlab/middleware/query_analyzer.rb:11:in `call'",
"lib/ci/job_token/middleware.rb:11:in `call'",
"lib/gitlab/middleware/multipart.rb:173:in `call'",
"lib/gitlab/middleware/read_only/controller.rb:50:in `call'",
"lib/gitlab/middleware/read_only.rb:18:in `call'",
"lib/gitlab/middleware/unauthenticated_session_expiry.rb:18:in `call'",
"lib/gitlab/middleware/secure_headers.rb:11:in `call'",
"lib/gitlab/middleware/same_site_cookies.rb:27:in `call'",
"lib/gitlab/middleware/path_traversal_check.rb:40:in `call'",
"lib/gitlab/middleware/handle_malformed_strings.rb:21:in `call'",
"lib/gitlab/middleware/basic_health_check.rb:25:in `call'",
"lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call'",
"lib/gitlab/middleware/request_context.rb:15:in `call'",
"lib/gitlab/middleware/webhook_recursion_detection.rb:15:in `call'",
"config/initializers/fix_local_cache_middleware.rb:11:in `call'",
"lib/gitlab/middleware/compressed_json.rb:44:in `call'",
"lib/gitlab/middleware/rack_multipart_tempfile_factory.rb:19:in `call'",
"lib/gitlab/metrics/requests_rack_middleware.rb:83:in `call'",
"lib/gitlab/middleware/sidekiq_web_static.rb:20:in `call'",
"lib/gitlab/middleware/release_env.rb:12:in `call'"
],
"exception.cause_class": "NoMethodError",
Affected Versions: GitLab 18.1 → 18.2 upgrades
Customer Impact: Major - Users unable to push code or access projects during upgrade process until post-deployment migrations complete.
Priority: High - Affects zero-downtime upgrades and requires backport to 18.2 version. Fixed in GitLab 18.4, requires backport to 18.2 for affected upgrade path.
References
- Related MR: !203295 (merged)
Edited by Kate Grechishkina