Target repository_languages rows by stable language_id in language detection updates and deletions

Related to #519895 (closed).

Problem

When language detection runs, Projects::DetectRepositoryLanguagesService updates and deletes repository_languages rows using values produced by Gitlab::LanguageDetection:

  • #updates targets rows by programming_language_id
  • #deletions returns programming_language_id values for delete_all

programming_languages.id is not stable across GitLab instances. On a cell that received migrated organizations, the copied repository_languages rows carry ids from the source cell, so updates silently match zero rows and deletions miss. Stale language data then persists forever, because detection is the mechanism that is supposed to correct it.

Proposal

Target rows by the stable language_id column, with a transitional fallback to the legacy id. The backfill of repository_languages.language_id can only be finalized in 19.6, so self-managed instances running 19.3–19.5 can still have NULL values that only the legacy id can match:

  • #updates: key row targeting on language_id, falling back to programming_language_id for rows where language_id is NULL.
  • #deletions: return the languages to delete so the service can match language_id OR the legacy programming_language_id.

This makes the write path self-healing after a cell move: the first detection run corrects rows that carry stale ids from the source cell.

Implementation notes

  • #insertions must continue to populate programming_language_id until the column is dropped as part of #614126.
  • The matching of existing rows by language name relies on the RepositoryLanguage#programming_language association; #614125 (closed) reroutes it through language_id, so this issue should land after it.

Follow-up in 19.6

See #614144 for removing the transitional legacy ID fallback after the backfill and NOT NULL validation are complete.

Edited by Vasilii Iakliushin