Add git audit streaming events

Issue Link: #343984 (closed)

Depend on

Issue: #351863 (closed)

What does this MR do and why?

Need a way to monitor who downloaded a certain repo, especially before the person leaves the job, it helps the administrator to discover risks.

The developer can download repo code through multiple protocols:

• git client by https/http
• git client by ssh
• download button on Web GUI

No matter which protocol the developer downloads the code through, we will send a notification.

The notification here is in the form of Audit Event Streaming !73149 (merged) .

Downloading the code of a public repository will not send a notification .

Precondition of the repository

• Ensure you are using GitLab Ultimate License.

# find the group of the repo
group = Group.find(123)

::Feature.enable(:audit_event_streaming_git_operations, group)

# Setup the external audit event destination
# It can also be set via graphql, see https://gitlab.com/gitlab-org/gitlab/-/merge_requests/73149#how-to-set-up-and-validate-locally
destination_url = 'https://icbd.requestcatcher.com/fetch-logs'
AuditEvents::ExternalAuditEventDestination.create(namespace_id: group.id, destination_url: destination_url)

We can view requests on https://icbd.requestcatcher.com/ :

Implement plan

Since requests for git repositories will eventually be handled by Gitaly, I tried to modify Gitaly at first. That is to say, when Gitaly receives a new git request, it sends a gRPC message back to GitLab Rails, and GitLab Rails send the Stream Audit Event.

There are two problems here. For the repository that doesn't need this feature, there are too many redundant messages. Furthermore, for Gitaly, we don't know the user information requested by git. Gitaly has the ability to accept user information, but the caller did not pass it. gitaly!2802 (merged)

Afterwards, we turned our attention to GitLab Rails.

All Git requests require authorization, through ProjectPolicy#access_allowed_to?.

But this judgment is too specific, we can't easily get the request details of git when intercepting it.

Finally, I Implemented it by intercepting the method on the Controller:

• git by https (Repositories::GitHttpController#git_upload_pack)
• git by ssh (API::Internal::Base, /api/v4/internal/allowed)
• download button on Web GUI (Projects::RepositoriesController#archive. Has been implemented by EE::Projects::RepositoriesController#log_audit_event)

Demo video

fetch_events