Skip to content

Fix reserved migration paths to work with hashed storage

From: https://gitlab.com/gitlab-com/infrastructure/issues/2821#note_47684308

In %10.0 we introduced "hashed storage". When enabled, rather than storing projects on disk in a location determined by project.full_path, they are now stored in @hashed/aa/bb/full-hash. The hash is generated by SHAing the project ID.

The use of full_path needs to be re-evalulated and tested. We now have Project#disk_path.

# We should modify the "rename reserved paths" migration so that it doesn't
# move files on disc for hashed projects.
#
# They were introduced some time ago but people migrating lots of releases at
# a time may well try to run this and hashed storage together
#
# Not all of these sites are bad, but they all need looking more closely at
lib/gitlab/database/rename_reserved_paths_migration/v1/migration_classes.rb:7:            def full_path
lib/gitlab/database/rename_reserved_paths_migration/v1/migration_classes.rb:9:                @full_path ||= route.path
lib/gitlab/database/rename_reserved_paths_migration/v1/migration_classes.rb:13:                build_full_path
lib/gitlab/database/rename_reserved_paths_migration/v1/migration_classes.rb:17:            def build_full_path
lib/gitlab/database/rename_reserved_paths_migration/v1/migration_classes.rb:19:                parent.full_path + '/' + path
lib/gitlab/database/rename_reserved_paths_migration/v1/migration_classes.rb:32:              route.path = build_full_path
lib/gitlab/database/rename_reserved_paths_migration/v1/migration_classes.rb:33:              @full_path = nil
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_base.rb:26:            old_full_path = routable.full_path
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_base.rb:28:            namespace_path = remove_last_occurrence(old_full_path, old_path)
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_base.rb:30:            new_full_path = join_routable_path(namespace_path, new_path)
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_base.rb:32:            perform_rename(routable, old_full_path, new_full_path)
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_base.rb:34:            [old_full_path, new_full_path]
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_base.rb:37:          def perform_rename(routable, old_full_path, new_full_path)
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_base.rb:39:            new_path = new_full_path.split('/').last
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_base.rb:43:            rename_routes(old_full_path, new_full_path)
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_base.rb:46:          def rename_routes(old_full_path, new_full_path)
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_base.rb:49:            quoted_old_full_path = quote_string(old_full_path)
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_base.rb:50:            quoted_old_wildcard_path = quote_string("#{old_full_path}/%")
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_base.rb:53:                       "lower(routes.path) = lower('#{quoted_old_full_path}') "\
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_base.rb:57:                       "( SELECT routes.id FROM routes WHERE lower(routes.path) = lower('#{quoted_old_full_path}') "\
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_base.rb:62:                                            old_full_path,
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_base.rb:63:                                            new_full_path)
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_base.rb:97:          def route_exists?(full_path)
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_base.rb:98:            MigrationClasses::Route.where(Route.arel_table[:path].matches(full_path)).any?
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_namespaces.rb:27:            old_full_path, new_full_path = rename_path_for_routable(namespace)
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_namespaces.rb:29:            track_rename('namespace', old_full_path, new_full_path)
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_namespaces.rb:31:            rename_namespace_dependencies(namespace, old_full_path, new_full_path)
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_namespaces.rb:34:          def rename_namespace_dependencies(namespace, old_full_path, new_full_path)
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_namespaces.rb:35:            move_repositories(namespace, old_full_path, new_full_path)
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_namespaces.rb:36:            move_uploads(old_full_path, new_full_path)
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_namespaces.rb:37:            move_pages(old_full_path, new_full_path)
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_namespaces.rb:38:            rename_user(old_full_path, new_full_path) if namespace.kind == 'user'
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_namespaces.rb:64:          def move_repositories(namespace, old_full_path, new_full_path)
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_namespaces.rb:67:              gitlab_shell.add_namespace(repository_storage_path, old_full_path)
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_namespaces.rb:69:              unless gitlab_shell.mv_namespace(repository_storage_path, old_full_path, new_full_path)
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_namespaces.rb:71:                           from #{old_full_path} to #{new_full_path}"
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_projects.rb:17:            old_full_path, new_full_path = rename_path_for_routable(project)
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_projects.rb:19:            track_rename('project', old_full_path, new_full_path)
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_projects.rb:21:            move_project_folders(project, old_full_path, new_full_path)
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_projects.rb:24:          def move_project_folders(project, old_full_path, new_full_path)
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_projects.rb:25:            move_repository(project, old_full_path, new_full_path)
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_projects.rb:26:            move_repository(project, "#{old_full_path}.wiki", "#{new_full_path}.wiki")
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_projects.rb:27:            move_uploads(old_full_path, new_full_path)
lib/gitlab/database/rename_reserved_paths_migration/v1/rename_projects.rb:28:            move_pages(old_full_path, new_full_path)
Edited by Stan Hu