Skip to content

Clean up expired agent activity events

Tiger Watson requested to merge 340364-create-agent-activity-events-with-cleanup into master

What does this MR do and why?

Creates a soft limit for the total number of activity events that can be associated with an agent. When an event is created for an agent, a cleanup job is scheduled to keep the number of events at the limit, starting with the oldest.

Events may be created frequently, so instead of scheduling a large number of Sidekiq jobs events are cleaned up at most once per agent per hour. This limit is enforced by deduplicating against jobs that are already in the queue with including_scheduled: true - a cleanup job is scheduled up to an hour in the future, and no further jobs can be enqueued (for that particular agent) until the first has executed.

Screenshots or screen recordings

These are strongly recommended to assist reviewers and reduce the time to merge your change.

How to set up and validate locally

In a Rails console:

  1. Create an agent record: agent = Clusters::Agent.create!(project: <project>, name: 'my-agent')
  2. Create several hundred activity events: 300.times { agent.activity_events.create!(kind: :agent_connected, level: :info, recorded_at: Time.now) }
  3. Execute the cleanup worker: Clusters::Agents::DeleteExpiredEventsWorker.new.perform(agent.id)
  4. Observe all but the most recent 200 events have been removed.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #340364 (closed)

Edited by Tiger Watson

Merge request reports