[Improved] Periodically save last activity date data from Redis to the database
What does this MR do?
-
It changes the way we store users last activity in Redis, from a Sorted Set to a Hash -
It adds a new users.last_activity_on
column-
Use it in the /user/activities
API instead of getting the data from Redis
-
-
It periodically (each day) schedules UpdateUserActivityWorker
which takes a batch of user activities from the Redis Hash (i.e.{ "USER_ID" => "TIMESTAMP" }
, usingScheduleUpdateUserActivityWorker
-
It updates users' last_activity_on
usingUpdateUserActivityWorker
with a single SQL query and removes the keys from the Redis Hash -
It adds a post-deploy migration to migrate the data from the Sorted Set to the new users.last_activity_on
column-
Test the migration with 400K records in the legacy user/activities
Sorted set
-
[1] pry(main)> Gitlab::Redis.with do |redis|
[1] pry(main)* redis.zcount('user/activities', 6.months.ago.to_i, Time.now.to_i)
[1] pry(main)* end
=> 455000
[2] pry(main)> User.where.not(last_activity_on: nil).count
(73.3ms) SELECT COUNT() FROM "users" WHERE ("users"."last_activity_on" IS NOT NULL)
=> 0
[3] pry(main)> User.where(last_activity_on: nil).count
(60.1ms) SELECT COUNT() FROM "users" WHERE "users"."last_activity_on" IS NULL
=> 457500
```
```
bin/rake db:migrate == 20170324160416 MigrateUserActivitiesToUsersLastActivityOn: migrating ======= == 20170324160416 MigrateUserActivitiesToUsersLastActivityOn: migrated (57.9876s) ```
-
Test the migration on MySQL -
It removes the lib/gitlab/user_activities/
code
Are there points in the code the reviewer needs to double check?
Does this MR meet the acceptance criteria?
-
Changelog entry added, if necessary -
Documentation created/updated -
API support added - Tests
-
Added for this feature/bug -
All builds are passing
-
-
Conform by the merge request performance guides -
Conform by the style guides -
Branch has no merge conflicts with master
(if it does - rebase it please) -
Squashed related commits together
What are the relevant issue numbers?
Closes #27790