Fix RedisHLL migration related to removal of custom slots

What does this MR do and why?

It was discovered that in !110952 (merged) there was a bug in Redis data migration script.

The data wasn't migrated because !110952 (diffs) wasn't actually loading events (called from the wrong place), events aren't stored in db/post_migration folder.

This MR re-introduces the migration with the bug being fixed.

At the moment we have data being added to Redis with new keys. Old keys are still in Redis. ServicePing aggregates data taking the new keys into account.

This MR merged old keys with new ones, so we don't loose the old data on Gitlab.com

Custom redis_slot will be removed a following MR, it's not in use in the current codebase but still required for migration.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

How to set up and validate locally

1. Add keys to Redis HLL from rails console:

> Gitlab::Redis::HLL.add(key: "users_viewing_{analytics}_group_devops_adoption-2023-14", value: 2, expiry: 6.weeks)
=> [true, true]
[46] pry(main)> Gitlab::Redis::HLL.count(keys: "users_viewing_{analytics}_group_devops_adoption-2023-14") # old key
=> 1 
[50] pry(main)> Gitlab::Redis::HLL.count(keys: "{hll_counters}_users_viewing_group_devops_adoption-2023-14") # new key
=> 0 # check that new keys is empty

2. re-run old broken migration bin/rails db:migrate:redo:main VERSION=20230202211434
3. make sure keys weren't migrated

> Gitlab::Redis::HLL.count(keys: "{hll_counters}_users_viewing_analytics_group_devops_adoption-2023-14")
=> 0

4. checkout the current branch git checkout 280527-remove-the-possibility-to-set-redis_slot-in-known_events
5. apply patch to make migration more visible:

Index: db/post_migrate/20230302811133_re_migrate_redis_slot_keys.rb
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/db/post_migrate/20230302811133_re_migrate_redis_slot_keys.rb b/db/post_migrate/20230328111013_re_migrate_redis_slot_keys.rb
rename from db/post_migrate/20230302811133_re_migrate_redis_slot_keys.rb
rename to db/post_migrate/20230328111013_re_migrate_redis_slot_keys.rb
--- a/db/post_migrate/20230302811133_re_migrate_redis_slot_keys.rb	(revision e11703a6376860cf8042aa73180bdd9e71d76afd)
+++ b/db/post_migrate/20230328111013_re_migrate_redis_slot_keys.rb	(date 1679990300856)
@@ -45,6 +45,7 @@
     Gitlab::Redis::SharedState.with do |r|
       break unless r.exists?(old_key)
 
+      pp "Renamed #{old_key} to #{new_key}"
       Gitlab::Redis::HLL.add(
         key: new_key,
         value: r.pfcount(old_key),

6. run the new migration bin/rails db:migrate:redo:main VERSION=20230328111013
7. make sure keys were migrated

> Gitlab::Redis::HLL.count(keys: "{hll_counters}_users_viewing_analytics_group_devops_adoption-2023-14")
=> 1

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.

Related to #280527 (closed)

changed milestone to %15.10

added devopsanalyze groupanalytics instrumentation quad-planningcomplete-action sectionanalytics typefeature workflowverification labels

assigned to @nbelokolodov

added 1 commit

• d07b26d8 - Migrate the existing RedisHLL keys to default slot

Compare with previous version

added database databasereview pending labels

	2 Warnings
	This merge request is quite big (732 lines changed), please consider splitting it into multiple merge requests.
	New migrations added but db/structure.sql wasn't updated Usually, when adding new migrations, db/structure.sql should be updated too (unless the migration isn't changing the DB schema and isn't the most recent one).

Reviewer roulette

Changes that require review have been detected!

Please refer to the table below for assigning reviewers and maintainers suggested by Danger in the specified category:

Category	Reviewer	Maintainer
backend	Thomas Hutterer (@thutterer) (UTC+2, 11 hours behind @nbelokolodov)	Kassio Borges (@kassio) (UTC+1, 12 hours behind @nbelokolodov)
database	Terri Chu (@terrichu) (UTC-4, 17 hours behind @nbelokolodov)	Mayra Cabrera (@mayra-cabrera) (UTC-6, 19 hours behind @nbelokolodov)
~"group::integrations" (backend)	Bojan Marjanović (@bmarjanovic) (UTC+2, 11 hours behind @nbelokolodov)	Maintainer review is optional for ~"group::integrations" (backend)
~"migration"	No reviewer available	No maintainer available
product intelligence	Mikołaj Wawrzyniak (@mikolaj_wawrzyniak) (UTC+0, 13 hours behind @nbelokolodov)	Maintainer review is optional for product intelligence

To spread load more evenly across eligible reviewers, Danger has picked a candidate for each review slot, based on their timezone. Feel free to override these selections if you think someone else would be better-suited or use the GitLab Review Workload Dashboard to find other available reviewers.

To read more on how to use the reviewer roulette, please take a look at the Engineering workflow and code review guidelines. Please consider assigning a reviewer or maintainer who is a domain expert in the area of the merge request.

Once you've decided who will review this merge request, assign them as a reviewer! Danger does not automatically notify them for you.

If needed, you can retry the danger-review job that generated this comment.

Generated by Danger

added 1 commit

• e11703a6 - Migrate the existing RedisHLL keys to default slot

Compare with previous version

changed title from Draft: Resolve "Remove the possibility to set redis_slot in known_events" to Draft: Remove the possibility to set redis_slot in known_events

changed the description

changed the description

changed the description

mentioned in issue #280527 (closed)

added 1 commit

• 717fb51f - Migrate the existing RedisHLL keys to default slot

Compare with previous version

marked the checklist item I have evaluated the MR acceptance checklist for this MR. as completed

added backend product intelligence product intelligencereview pending labels

added Data WarehouseImpact Check label

added 1 commit

• 451fc527 - Migrate the existing RedisHLL keys to default slot

Compare with previous version

changed the description

changed the description

changed title from Draft: Remove the possibility to set redis_slot in known_events to Draft: Fix RedisHLL migration related to removal of custom slots

marked this merge request as ready

added 1 commit

• 841db9b1 - Migrate the existing RedisHLL keys to default slot

Compare with previous version

added bugfunctional typebug + 1 deleted label and removed typefeature label

changed milestone to %15.11

@mikolaj_wawrzyniak could you please take over backend and product intelligence review?

requested review from @mikolaj_wawrzyniak

added 1 commit

• d1f0a4d9 - Migrate Redis HLL keys with merge

Compare with previous version

removed workflowverification label

requested review from @terrichu and @michold and removed review request for @mikolaj_wawrzyniak

assigned to @mikolaj_wawrzyniak

approved this merge request

added product intelligenceapproved label and removed product intelligencereview pending label

requested review from @kerrizor and removed review request for @michold

@michold, thanks for approving this merge request.

This is the first time the merge request is approved. To ensure full test coverage, a new pipeline will be started shortly.

For more info, please refer to the following links:

• rspec predictive jobs
• jest predictive jobs
• merging a merge request.

added pipeline:mr-approved label

added severity1 label

added 238 commits

• d1f0a4d9...2735f542 - 237 commits from branch master
• 53264d58 - Merge branch 'master' into '280527-remove-the-possibility-to-set-redis_slot-in-known_events'

Compare with previous version

removed review request for @kerrizor

requested review from @kerrizor

approved this merge request

approved this merge request

added databasereviewed label and removed databasereview pending label

requested review from @a_akgun and removed review request for @terrichu