Group transfer failure via API when group has archived projects - find_or_create_label method fails to create label

Summary

@kategrechishkina and I received a complaint from a customer because they found errors while trying to migrate a group. Upon further investigation we discovered that the transfer was failing because there were archived projects within the group being transferred, and some of the labels inside the parent group were assigned to those archived projects.

Here's the stack trace of the error:

Click to expand

{ "severity": "ERROR", "time": "2025-07-02T13:30:36.780Z", "correlation_id": "[REDACTED]", "meta.caller_id": "POST /api/:version/groups/:id/transfer", "meta.remote_ip": "[REDACTED]", "meta.feature_category": "groups_and_projects", "meta.user": "[REDACTED]", "meta.user_id": [REDACTED], "meta.root_namespace": "[REDACTED]", "meta.client_id": "[REDACTED]", "meta.organization_id": 1, "exception.class": "NoMethodError", "exception.message": "undefined method \`id' for nil:NilClass", "exception.backtrace": \[ "app/services/labels/transfer_service.rb:69:in \`find_or_create_label!'", "app/services/labels/transfer_service.rb:24:in \`block (2 levels) in execute'", "activerecord (7.0.8.7) lib/active_record/relation/batches.rb:71:in \`each'", "activerecord (7.0.8.7) lib/active_record/relation/batches.rb:71:in \`block in find_each'", "activerecord (7.0.8.7) lib/active_record/relation/batches.rb:138:in \`block in find_in_batches'", "activerecord (7.0.8.7) lib/active_record/relation/batches.rb:245:in \`block in in_batches'", "activerecord (7.0.8.7) lib/active_record/relation/batches.rb:229:in \`loop'", "activerecord (7.0.8.7) lib/active_record/relation/batches.rb:229:in \`in_batches'", "activerecord (7.0.8.7) lib/active_record/relation/batches.rb:137:in \`find_in_batches'", "activerecord (7.0.8.7) lib/active_record/relation/batches.rb:70:in \`find_each'", "app/services/labels/transfer_service.rb:23:in \`block in execute'", "app/models/concerns/cross_database_modification.rb:91:in \`block in transaction'", "activerecord (7.0.8.7) lib/active_record/connection_adapters/abstract/transaction.rb:319:in \`block in within_new_transaction'", "activesupport (7.0.8.7) lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in \`handle_interrupt'", "activesupport (7.0.8.7) lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in \`block in synchronize'", "activesupport (7.0.8.7) lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in \`handle_interrupt'", "activesupport (7.0.8.7) lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in \`synchronize'", "activerecord (7.0.8.7) lib/active_record/connection_adapters/abstract/transaction.rb:317:in \`within_new_transaction'", "activerecord (7.0.8.7) lib/active_record/connection_adapters/abstract/database_statements.rb:316:in \`transaction'", "lib/gitlab/database/load_balancing/connection_proxy.rb:127:in \`public_send'", "lib/gitlab/database/load_balancing/connection_proxy.rb:127:in \`block in write_using_load_balancer'", "lib/gitlab/database/load_balancing/load_balancer.rb:141:in \`block in read_write'", "lib/gitlab/database/load_balancing/load_balancer.rb:235:in \`retry_with_backoff'", "lib/gitlab/database/load_balancing/load_balancer.rb:130:in \`read_write'", "lib/gitlab/database/load_balancing/connection_proxy.rb:126:in \`write_using_load_balancer'", "lib/gitlab/database/load_balancing/connection_proxy.rb:78:in \`transaction'", "activerecord (7.0.8.7) lib/active_record/transactions.rb:209:in \`transaction'", "lib/gitlab/database.rb:399:in \`block in transaction'", "activesupport (7.0.8.7) lib/active_support/notifications.rb:206:in \`block in instrument'", "activesupport (7.0.8.7) lib/active_support/notifications/instrumenter.rb:24:in \`instrument'", "activesupport (7.0.8.7) lib/active_support/notifications.rb:206:in \`instrument'", "lib/gitlab/database.rb:398:in \`transaction'", "app/models/concerns/cross_database_modification.rb:82:in \`transaction'", "app/services/labels/transfer_service.rb:22:in \`execute'", "app/services/groups/transfer_service.rb:87:in \`block (2 levels) in transfer_labels'", "activerecord (7.0.8.7) lib/active_record/relation/delegation.rb:88:in \`each'", "activerecord (7.0.8.7) lib/active_record/relation/delegation.rb:88:in \`each'", "app/services/groups/transfer_service.rb:86:in \`block in transfer_labels'", "app/models/concerns/each_batch.rb:102:in \`block (2 levels) in each_batch'", "activerecord (7.0.8.7) lib/active_record/relation.rb:881:in \`\_scoping'", "activerecord (7.0.8.7) lib/active_record/relation.rb:428:in \`scoping'", "activerecord (7.0.8.7) lib/active_record/scoping/default.rb:43:in \`unscoped'", "app/models/concerns/each_batch.rb:102:in \`block in each_batch'", "app/models/concerns/each_batch.rb:72:in \`step'", "app/models/concerns/each_batch.rb:72:in \`each_batch'", "activerecord (7.0.8.7) lib/active_record/relation/delegation.rb:108:in \`public_send'", "activerecord (7.0.8.7) lib/active_record/relation/delegation.rb:108:in \`block in method_missing'", "activerecord (7.0.8.7) lib/active_record/relation.rb:881:in \`\_scoping'", "activerecord (7.0.8.7) lib/active_record/relation.rb:428:in \`scoping'", "activerecord (7.0.8.7) lib/active_record/relation/delegation.rb:108:in \`method_missing'", "app/services/groups/transfer_service.rb:85:in \`transfer_labels'", "app/services/groups/transfer_service.rb:75:in \`proceed_to_transfer'", "app/services/groups/transfer_service.rb:25:in \`execute'", "lib/api/groups.rb:607:in \`block (2 levels) in '", "grape (2.0.0) lib/grape/endpoint.rb:58:in \`call'", "grape (2.0.0) lib/grape/endpoint.rb:58:in \`block (2 levels) in generate_api_method'", "activesupport (7.0.8.7) lib/active_support/notifications.rb:208:in \`instrument'", "grape (2.0.0) lib/grape/endpoint.rb:57:in \`block in generate_api_method'", "grape (2.0.0) lib/grape/endpoint.rb:328:in \`execute'", "grape (2.0.0) lib/grape/endpoint.rb:260:in \`block in run'", "activesupport (7.0.8.7) lib/active_support/notifications.rb:208:in \`instrument'", "grape (2.0.0) lib/grape/endpoint.rb:240:in \`run'", "grape (2.0.0) lib/grape/endpoint.rb:316:in \`block in build_stack'", "grape (2.0.0) lib/grape/middleware/base.rb:36:in \`call!'", "grape (2.0.0) lib/grape/middleware/base.rb:29:in \`call'", "grape (2.0.0) lib/grape/middleware/base.rb:36:in \`call!'", "grape (2.0.0) lib/grape/middleware/base.rb:29:in \`call'", "ee/lib/gitlab/middleware/ip_restrictor.rb:14:in \`block in call'", "lib/gitlab/ip_address_state.rb:11:in \`with'", "ee/lib/gitlab/middleware/ip_restrictor.rb:13:in \`call'", "grape (2.0.0) lib/grape/middleware/base.rb:36:in \`call!'", "grape (2.0.0) lib/grape/middleware/base.rb:29:in \`call'", "lib/api/api_guard.rb:247:in \`call'", "grape (2.0.0) lib/grape/middleware/base.rb:36:in \`call!'", "grape (2.0.0) lib/grape/middleware/base.rb:29:in \`call'", "rack-oauth2 (2.2.1) lib/rack/oauth2/server/resource.rb:20:in \`\_call'", "rack-oauth2 (2.2.1) lib/rack/oauth2/server/resource/bearer.rb:8:in \`\_call'", "rack-oauth2 (2.2.1) lib/rack/oauth2/server/abstract/handler.rb:17:in \`call'", "grape (2.0.0) lib/grape/middleware/error.rb:39:in \`block in call!'", "grape (2.0.0) lib/grape/middleware/error.rb:38:in \`catch'", "grape (2.0.0) lib/grape/middleware/error.rb:38:in \`call!'", "grape (2.0.0) lib/grape/middleware/base.rb:29:in \`call'", "grape_logging (1.8.4) lib/grape_logging/middleware/request_logger.rb:60:in \`block in call!'", "grape_logging (1.8.4) lib/grape_logging/middleware/request_logger.rb:58:in \`catch'", "grape_logging (1.8.4) lib/grape_logging/middleware/request_logger.rb:58:in \`call!'", "grape (2.0.0) lib/grape/middleware/base.rb:29:in \`call'", "rack (2.2.13) lib/rack/head.rb:12:in \`call'", "grape (2.0.0) lib/grape/endpoint.rb:224:in \`call!'", "grape (2.0.0) lib/grape/endpoint.rb:218:in \`call'", "grape (2.0.0) lib/grape/router/route.rb:58:in \`exec'", "grape (2.0.0) lib/grape/router.rb:120:in \`process_route'", "grape (2.0.0) lib/grape/router.rb:74:in \`block in identity'", "grape (2.0.0) lib/grape/router.rb:94:in \`transaction'", "grape (2.0.0) lib/grape/router.rb:72:in \`identity'", "grape (2.0.0) lib/grape/router.rb:56:in \`block in call'", "grape (2.0.0) lib/grape/router.rb:136:in \`with_optimization'", "grape (2.0.0) lib/grape/router.rb:55:in \`call'", "grape (2.0.0) lib/grape/api/instance.rb:165:in \`call'", "grape (2.0.0) lib/grape/api/instance.rb:70:in \`call!'", "grape (2.0.0) lib/grape/api/instance.rb:65:in \`call'", "actionpack (7.0.8.7) lib/action_dispatch/routing/mapper.rb:19:in \`block in '", "actionpack (7.0.8.7) lib/action_dispatch/routing/mapper.rb:48:in \`serve'", "actionpack (7.0.8.7) lib/action_dispatch/journey/router.rb:50:in \`block in serve'", "actionpack (7.0.8.7) lib/action_dispatch/journey/router.rb:32:in \`each'", "actionpack (7.0.8.7) lib/action_dispatch/journey/router.rb:32:in \`serve'", "actionpack (7.0.8.7) lib/action_dispatch/routing/route_set.rb:852:in \`call'", "gitlab-experiment (0.9.1) lib/gitlab/experiment/middleware.rb:19:in \`call'", "omniauth (2.1.2) lib/omniauth/strategy.rb:202:in \`call!'", "omniauth (2.1.2) lib/omniauth/strategy.rb:169:in \`call'", "omniauth (2.1.2) lib/omniauth/strategy.rb:202:in \`call!'", "omniauth (2.1.2) lib/omniauth/strategy.rb:169:in \`call'", "flipper (0.28.3) lib/flipper/middleware/memoizer.rb:72:in \`memoized_call'", "flipper (0.28.3) lib/flipper/middleware/memoizer.rb:37:in \`call'", "lib/gitlab/metrics/elasticsearch_rack_middleware.rb:16:in \`call'", "lib/gitlab/middleware/sidekiq_shard_awareness_validation.rb:20:in \`block in call'", "lib/gitlab/sidekiq_sharding/validator.rb:42:in \`enabled'", "lib/gitlab/middleware/sidekiq_shard_awareness_validation.rb:20:in \`call'", "lib/gitlab/middleware/memory_report.rb:13:in \`call'", "lib/gitlab/middleware/speedscope.rb:13:in \`call'", "lib/gitlab/database/load_balancing/rack_middleware.rb:23:in \`call'", "lib/gitlab/middleware/rails_queue_duration.rb:33:in \`call'", "lib/gitlab/etag_caching/middleware.rb:21:in \`call'", "lib/gitlab/metrics/rack_middleware.rb:16:in \`block in call'", "lib/gitlab/metrics/web_transaction.rb:46:in \`run'", "lib/gitlab/metrics/rack_middleware.rb:16:in \`call'", "lib/gitlab/middleware/go.rb:21:in \`call'", "lib/gitlab/middleware/query_analyzer.rb:11:in \`block in call'", "lib/gitlab/database/query_analyzer.rb:83:in \`within'", "lib/gitlab/middleware/query_analyzer.rb:11:in \`call'", "lib/ci/job_token/middleware.rb:11:in \`call'", "batch-loader (2.0.5) lib/batch_loader/middleware.rb:11:in \`call'", "rack-attack (6.7.0) lib/rack/attack.rb:103:in \`call'", "apollo_upload_server (2.1.6) lib/apollo_upload_server/middleware.rb:19:in \`call'", "lib/gitlab/middleware/multipart.rb:173:in \`call'", "rack-attack (6.7.0) lib/rack/attack.rb:127:in \`call'", "warden (1.2.9) lib/warden/manager.rb:36:in \`block in call'", "warden (1.2.9) lib/warden/manager.rb:34:in \`catch'", "warden (1.2.9) lib/warden/manager.rb:34:in \`call'", "rack-cors (2.0.2) lib/rack/cors.rb:102:in \`call'", "rack (2.2.13) lib/rack/tempfile_reaper.rb:15:in \`call'", "rack (2.2.13) lib/rack/etag.rb:27:in \`call'", "rack (2.2.13) lib/rack/conditional_get.rb:40:in \`call'", "rack (2.2.13) lib/rack/head.rb:12:in \`call'", "actionpack (7.0.8.7) lib/action_dispatch/http/permissions_policy.rb:38:in \`call'", "actionpack (7.0.8.7) lib/action_dispatch/http/content_security_policy.rb:39:in \`call'", "lib/gitlab/middleware/read_only/controller.rb:50:in \`call'", "lib/gitlab/middleware/read_only.rb:18:in \`call'", "lib/gitlab/middleware/unauthenticated_session_expiry.rb:18:in \`call'", "rack (2.2.13) lib/rack/session/abstract/id.rb:266:in \`context'", "rack (2.2.13) lib/rack/session/abstract/id.rb:260:in \`call'", "lib/gitlab/middleware/secure_headers.rb:11:in \`call'", "actionpack (7.0.8.7) lib/action_dispatch/middleware/cookies.rb:704:in \`call'", "lib/gitlab/middleware/same_site_cookies.rb:27:in \`call'", "actionpack (7.0.8.7) lib/action_dispatch/middleware/callbacks.rb:27:in \`block in call'", "activesupport (7.0.8.7) lib/active_support/callbacks.rb:99:in \`run_callbacks'", "actionpack (7.0.8.7) lib/action_dispatch/middleware/callbacks.rb:26:in \`call'", "sentry-rails (5.22.1) lib/sentry/rails/rescued_exception_interceptor.rb:14:in \`call'", "actionpack (7.0.8.7) lib/action_dispatch/middleware/debug_exceptions.rb:28:in \`call'", "lib/gitlab/middleware/path_traversal_check.rb:35:in \`call'", "lib/gitlab/middleware/handle_malformed_strings.rb:21:in \`call'", "sentry-ruby (5.22.1) lib/sentry/rack/capture_exceptions.rb:30:in \`block (2 levels) in call'", "sentry-ruby (5.22.1) lib/sentry/hub.rb:269:in \`with_session_tracking'", "sentry-ruby (5.22.1) lib/sentry-ruby.rb:416:in \`with_session_tracking'", "sentry-ruby (5.22.1) lib/sentry/rack/capture_exceptions.rb:21:in \`block in call'", "sentry-ruby (5.22.1) lib/sentry/hub.rb:59:in \`with_scope'", "sentry-ruby (5.22.1) lib/sentry-ruby.rb:396:in \`with_scope'", "sentry-ruby (5.22.1) lib/sentry/rack/capture_exceptions.rb:20:in \`call'", "actionpack (7.0.8.7) lib/action_dispatch/middleware/show_exceptions.rb:29:in \`call'", "lib/gitlab/middleware/basic_health_check.rb:25:in \`call'", "lograge (0.11.2) lib/lograge/rails_ext/rack/logger.rb:15:in \`call_app'", "railties (7.0.8.7) lib/rails/rack/logger.rb:25:in \`block in call'", "activesupport (7.0.8.7) lib/active_support/tagged_logging.rb:99:in \`block in tagged'", "activesupport (7.0.8.7) lib/active_support/tagged_logging.rb:37:in \`tagged'", "activesupport (7.0.8.7) lib/active_support/tagged_logging.rb:99:in \`tagged'", "railties (7.0.8.7) lib/rails/rack/logger.rb:25:in \`call'", "actionpack (7.0.8.7) lib/action_dispatch/middleware/remote_ip.rb:93:in \`call'", "lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in \`call'", "lib/gitlab/middleware/request_context.rb:15:in \`call'", "lib/gitlab/middleware/webhook_recursion_detection.rb:15:in \`call'", "request_store (1.7.0) lib/request_store/middleware.rb:19:in \`call'", "rack (2.2.13) lib/rack/method_override.rb:24:in \`call'", "rack (2.2.13) lib/rack/runtime.rb:22:in \`call'", "rack-timeout (0.7.0) lib/rack/timeout/core.rb:154:in \`block in call'", "rack-timeout (0.7.0) lib/rack/timeout/support/timeout.rb:19:in \`timeout'", "rack-timeout (0.7.0) lib/rack/timeout/core.rb:153:in \`call'", "config/initializers/fix_local_cache_middleware.rb:11:in \`call'", "lib/gitlab/middleware/compressed_json.rb:44:in \`call'", "actionpack (7.0.8.7) lib/action_dispatch/middleware/executor.rb:14:in \`call'", "lib/gitlab/middleware/rack_multipart_tempfile_factory.rb:19:in \`call'", "rack (2.2.13) lib/rack/sendfile.rb:110:in \`call'", "lib/gitlab/middleware/sidekiq_web_static.rb:20:in \`call'", "lib/gitlab/metrics/requests_rack_middleware.rb:83:in \`call'", "gitlab-labkit (0.37.0) lib/labkit/middleware/rack.rb:22:in \`block in call'", "gitlab-labkit (0.37.0) lib/labkit/context.rb:35:in \`with_context'", "gitlab-labkit (0.37.0) lib/labkit/middleware/rack.rb:21:in \`call'", "actionpack (7.0.8.7) lib/action_dispatch/middleware/request_id.rb:26:in \`call'", "actionpack (7.0.8.7) lib/action_dispatch/middleware/host_authorization.rb:131:in \`call'", "railties (7.0.8.7) lib/rails/engine.rb:530:in \`call'", "railties (7.0.8.7) lib/rails/railtie.rb:226:in \`public_send'", "railties (7.0.8.7) lib/rails/railtie.rb:226:in \`method_missing'", "lib/gitlab/middleware/release_env.rb:12:in \`call'", "rack (2.2.13) lib/rack/urlmap.rb:74:in \`block in call'", "rack (2.2.13) lib/rack/urlmap.rb:58:in \`each'", "rack (2.2.13) lib/rack/urlmap.rb:58:in \`call'", "puma (6.5.0) lib/puma/configuration.rb:279:in \`call'", "puma (6.5.0) lib/puma/request.rb:99:in \`block in handle_request'", "puma (6.5.0) lib/puma/thread_pool.rb:389:in \`with_force_shutdown'", "puma (6.5.0) lib/puma/request.rb:98:in \`handle_request'", "puma (6.5.0) lib/puma/server.rb:468:in \`process_client'", "puma (6.5.0) lib/puma/server.rb:249:in \`block in run'", "puma (6.5.0) lib/puma/thread_pool.rb:166:in \`block in spawn_thread'" \], "user.username": "[REDACTED]", "tags.program": "web", "tags.locale": "en", "tags.feature_category": "groups_and_projects", "tags.correlation_id": "[REDACTED]" }

The stack trace is ultimately pointing to the new_label being returned as nil:

    def find_or_create_label!(label)
      params    = label.attributes.slice('title', 'description', 'color')
      new_label = FindOrCreateService.new(current_user, project, params.merge(include_ancestor_groups: true)).execute

      new_label.id
    end

If new_label is nil, it means that the line below returned nil for some reason:

FindOrCreateService.new(current_user, project, params.merge(include_ancestor_groups: true)).execute

Going inside the execute method, we find that it is calling the find_or_create_label method:

    # Only creates the label if current_user can do so, if the label does not exist
    # and the user can not create the label, nil is returned
    def find_or_create_label(find_only: false)
      new_label = find_existing_label(title)

      return new_label if find_only

      if new_label.nil? && (skip_authorization || Ability.allowed?(current_user, :admin_label, parent))
        create_params = params.except(:include_ancestor_groups)
        new_label = Labels::CreateService.new(create_params).execute(parent_type.to_sym => parent)
      end

      new_label
    end

Now, because the project is archived and is in read-only mode, the statement below returns false , making the label not be created:

Ability.allowed?(current_user, :admin_label, parent)

The label object being nil , all method calls on it fail, including new_label.id.

Also, one of the problems of this bug is that the transfer method (proceed_to_transfer) stops in the middle of its execution, meaning that some parts of the code are not executed and the group transfer doesn't finish. The bug happens in transfer_labels so the following methods are not executed:

      remove_paid_features_for_projects(old_root_ancestor_id)
      post_update_hooks(@updated_project_ids, old_root_ancestor_id)
      propagate_integrations
      update_pending_builds

Steps to reproduce

Have 2 top-level groups ready, for example: top_level_group_a and top_level_group_b
Create a project under a subgroup, for example:
- top_level_group_a
  - subgroup_b
    - project_c
- top_level_group_b
Create a label on top_level_group_a and assign it to an issue on project_c
Archive project_c
Now transfer subgroup_b to top_level_group_b
You should get a 500 error

Example Group

@kategrechishkina has already tested this with this group (GitLab Team Members Only). Please ping us to get access to it

What is the current bug behavior?

The bug happens when there are archived projects in the group being transferred, and labels from parent groups are assigned to those archived projects.

What is the expected correct behavior?

The transfer should continue without errors, but a decision has to be made regarding the expected behavior for archived projects.
- Should we give users a warning that any possible write operation will not be made when there are archived (read-only) projects in the group?
- Should we not transfer archived projects?
- Should we unarchive the project, transfer and archive again automatically?
- Should we block the transfer when there are archived projects in the group?
- I have no idea, but I guess this is where @lohrc comes in to save the day 🤩

Output of checks

This bug happens on GitLab.com

Possible fixes

This is the part of the code that is making the label return as nil:

    def find_or_create_label(find_only: false)
      new_label = find_existing_label(title)

      return new_label if find_only

      if new_label.nil? && (skip_authorization || Ability.allowed?(current_user, :admin_label, parent))
        create_params = params.except(:include_ancestor_groups)
        new_label = Labels::CreateService.new(create_params).execute(parent_type.to_sym => parent)
      end

      new_label
    end

Workaround

The current workaround is to unarchive all projects under the group before the transfer. However, this doesn't fix/complete transfers that have already failed.

Edited Jul 18, 2025 by 🤖 GitLab Bot 🤖