Improve `InternalId.track_greatest` for imported projects
Problem to solve
During import we perform track_greatest
of internal_id
.
This is to ensure that internal_id
table holds
the greatest ever used ID by the entries in database.
The Internal ID is being used to have a sequential numbers in context of the project,
so issues for project start with 1
, instead of 1_000_000
.
It seems that there are multiple things to consider:
- We always run
.track_greatest
for eachiid
recorded during import, - We seem to restore objects in reverse order (from the biggest),
- We seem to run
Project#after_import
=>InternalId.flush_records!(project: self)
that seems to resolve all internal_ids entries,
This seems that we can optimise that easily, by either:
- If the
.flush_records
is desired, we should ignore all internal id tracking code during import, as this is fully redundant, - If the
.flush_records
is not desired, we should cache in memory the tracket greatest value instead of checking that against database.
The check against database looks as this:
D, [2019-12-09T17:09:39.322184 #64] DEBUG -- : InternalId Create (0.4ms) INSERT INTO "internal_ids" ("project_id", "usage", "last_value") VALUES ($1, $2, $3) RETURNING "id" [["project_id", 43], ["usage", 1], ["last_value", 0]]
D, [2019-12-09T17:09:39.322253 #64] DEBUG -- : ↳ app/models/internal_id.rb:200
D, [2019-12-09T17:09:39.322548 #64] DEBUG -- : (0.1ms) RELEASE SAVEPOINT active_record_1
D, [2019-12-09T17:09:39.322628 #64] DEBUG -- : ↳ lib/gitlab/database.rb:301
D, [2019-12-09T17:09:39.323256 #64] DEBUG -- : InternalId Load (0.3ms) SELECT "internal_ids".* FROM "internal_ids" WHERE "internal_ids"."id" = $1 LIMIT $2 FOR UPDATE [["id", 97], ["LIMIT", 1]]
D, [2019-12-09T17:09:39.323323 #64] DEBUG -- : ↳ app/models/internal_id.rb:50
D, [2019-12-09T17:09:39.324285 #64] DEBUG -- : InternalId Update (0.4ms) UPDATE "internal_ids" SET "last_value" = $1 WHERE "internal_ids"."id" = $2 [["last_value", 1], ["id", 97]]
D, [2019-12-09T17:09:39.324342 #64] DEBUG -- : ↳ config/initializers/config_initializers_active_record_locking.rb:13