Pre-migration migrations block upgrade when using more than one gitlab instance

Hi, We are running gitlab on multiple nodes, load balanced. Gitlab itself handles it perfectly, but the registry brakes upgrade as :

  • First node runs a reconfigure action to check everything is fine before upgrading. This triggers a registry database migrate up which does nothing as we're up-to-date
  • First node run the pre-migration update, and updates itself
  • Second node runs a reconfigure action to check everything is fine before upgrading. This breaks with error likes failed to prepare Up plan: Unable to create migration plan because of 20250904041325_add_batching_strategy_and_total_tuple_count: unknown migration in database. This is because :
    • The second node still uses the N-1 version of registry binary
    • The migrations are hardcoded in binary
    • First node completed migration
    • So the second node finds migrations in DB which does not exists in its binary

The only workaround we found is to copy the new binary to second node, so that the registry database migrate up command can succeed.

Looks like the registry database migrate up should not fail when unknown migrations exists in database ?

Assignee Loading
Time tracking Loading