Skip to content

Add automation rule model and dispatch service

Tao Guo requested to merge taoza/automation/add-automation-model into master

What does this MR do and why?

This MR introduces a new model for storing automation rules and the associated event subscription as part of the No-code Automation feature developed by the Incubation Engineering org.

Following up on the conversation from !108956 (closed), this MR contains an alternative approach that uses a dedicated table for the required storage and subscription purposes. It aims at achieving a cleaner separation of concern by allowing some level of duplication.

To avoid complete duplication when implementing the event subscription mechanism, Automation::Rule makes use of TriggerableHooks's existing capability for filtering hooks based on the input events, e.g. issues_events. Based on Automation's MVC scope, it only needs to support issues and merge-requests events.

What does the new DB table do?

The new table essentially answers the question:

When an event occurs in GitLab (e.g. a new issue is created), which automation rule shall be executed?

This table is also the main source of the truth for the automation's management UI:

Screen_Shot_2023-01-09_at_1.33.51_pm

Columns breakdown

Field Description
namespace A foreign key that helps to associate automation rules with namespaces
issues_events A boolean field to indicate if the rule should be triggered by issue events such as create or update
merge_requests_events A boolean field to indicate if the rule should be triggered by MR events
permanently_disabled A boolean field used to disable the rule execution permanently. The choice of permanently_ prefix leaves room to support temporary disablement in the future.
name The name of the automation rule.
rule The rule definition serialized as YAML

Tables Risk

What is the anticipated growth for the new table over the next 3 months, 6 months, 1 year? What assumptions are these based on?

  • This a brand-new feature but much anticipated. I'm planning to put a limit on each project to have max 5 rules. With the goal to hit 500 projects in the next 12 months, that would translate to 6,000ish records.

How many reads and writes per hour would you expect this table to have in 3 months, 6 months, 1 year? Under what circumstances are rows updated? What assumptions are these based on?

  • The write to the table is triggered by pipeline execution and only when there is an update to the policy file in the repo. Based on the above numbers, I don't expect more than 1 write per minute.

Based on the anticipated data volume and access patterns, does the new table pose an availability risk to GitLab.com or self-managed instances? Does the proposed design scale to support the needs of GitLab.com and self-managed customers?

  • Yes, I'm confident with the current design and the potential for optimization as the traffic grow with this brand new feature.

Todo

  • Rebase master to squash cherry-picked feature category file once !108692 (merged) is merged.

How to set up and validate locally

  1. Run Rails console, e.g.
gdk rails c 
  1. Tail application log:
tail -f log/application.log | grep automation
  1. Run script below and observe the log output
# Pick the desired project an issue
project = Project.first
issue = project.issues.first

# Turn on feature flag
::Feature.enable(:no_code_automation_mvc, project)

# Create a new automation rule and subscribe to issue events
project.project_namespace.automation_rules.create!(name: "rule-#{Time.now}", rule: '-', issues_events: true)

# Trigger the automation rule by updating the issue
Issues::UpdateService.new(project: project, current_user: User.first, params: {title: "test-#{Time.now}"}).execute(issue)
  1. The log should output Placeholder for performing automation rules if successful.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Tao Guo

Merge request reports