Remove feature exclusion when writing audit events
Decision: Not planned at this time
Problem to solve
We are currently recording more information on audit events for higher tier customer. This approach has posed a number challenges:
- Upgrading to higher tier doesn't give customer instant access to these extra information on past audit events
- A shared DB schema for lower tiers might result in under-utilized fields or indexes (i.e. with NULL values)
- Complex write logic in application
Beside, removing this restriction will make it easier for us to move features down in future iterations.
Intended users
User experience goal
User can view these extra information on audit events when upgrading to higher tier.
Proposal
Record all information regardless of license when writing audit events but restricting on view.
We should add the following fields that are currently only available on GitLab Premium
ip_address-
entity_path(e.g.gitlab-org/gitlab-test) - impersonation information
Further details
Analysis on data storage
- For every record with the above extra information:
+100 bytes - Total number of audit events on
GitLab.com:319 M records(according to this report) - Total extra storage required:
319 M x 100 bytes = 29.7092 GiB
The current DB size is 78.85 GiB so this is going to be a 37.67% increase in storage.
Calculation
A full event
gitlabhq_development=# select * from audit_events where id=27;
details
--------------------------------------------------------
--- +
:change: prevent merge request approval from reviewers +
:from: +
:to: true +
:author_name: Gianna Spencer +
:target_id: 20 +
:target_type: Project +
:target_details: reported_user_20/awesome-swift +
:ip_address: 127.0.0.1 +
:entity_path: reported_user_20/awesome-swift +
:impersonated_by: Administrator +gitlabhq_development=# select pg_size_pretty(sum(pg_column_size(audit_events))), pg_size_pretty(sum(pg_column_size(details))) from audit_events where id=27;
pg_size_pretty | pg_size_pretty
----------------+----------------
384 bytes | 295 bytes A trimmed-down event
gitlabhq_development=# select details from audit_events where id=27;
details
--------------------------------------------------------
--- +
:change: prevent merge request approval from reviewers+
:from: +
:to: true +
:author_name: Gianna Spencer +
:target_id: 20 +
:target_type: Project +
:target_details: reported_user_20/awesome-swift +gitlabhq_development=# select pg_size_pretty(sum(pg_column_size(audit_events))), pg_size_pretty(sum(pg_column_size(details))) from audit_events where id=27;
pg_size_pretty | pg_size_pretty
----------------+----------------
280 bytes | 195 bytesPermissions and Security
Documentation
Availability & Testing
What does success look like, and how can we measure that?
What is the type of buyer?
Is this a cross-stage feature?
Links / references
Edited by Sam Kerr