Integrate audit event streaming with 3rd-party storage systems
# Problem
[Streaming Audit Events](https://docs.gitlab.com/ee/administration/audit_event_streaming.html) today are sent over HTTPS to a destination of a user's choosing. This is great because it offers a lot of flexibility. It also means that users must create their own logic to ingest events, clean, process, and store them before they can be meaningfully analyzed. This can cause frustration and a longer adoption time for users who already use 3rd party [SIEMs](https://en.wikipedia.org/wiki/Security_information_and_event_management) and systems for their data collection, analysis and storage.
# Proposal
Offer additional endpoint options for streaming audit events to send the data directly to the 3rd party systems that customers are already using.
## Data formats
Utilize JSON or [Parquet files](https://parquet.apache.org/) for storing the data.
* Parquet is a common format that many tools, such as [ClickHouse](https://clickhouse.com/), can read directly.
* _Note:_ Engineering input needed on this.
## Ordering
Many SIEMs today integrate directly with storage solutions such as [Amazon S3](https://aws.amazon.com/s3/), [Google Cloud Storage](https://cloud.google.com/storage), and [Azure Blob Storage](https://azure.microsoft.com/en-us/services/storage/blobs/). This represents an opportunity for us to get compability with a large number of SIEMs at once by integrating those storage systems, rather than focusing on individual SIEMs or other 3rd-party products.
In the future, we might consider integrating directly with a 3rd-party product if there is some additional value we can provide with a specific integration, but will prioritize those storage solutions first.
## High-level requirements
1. All data available over the existing HTTPS system should be available if the users chooses a different endpoint type.
1. User credentials for authenticating to external storage systems should be stored securely and leverage existing GitLab secure credential systems where possible, rather than building something new.
1. Solutions should be built with APIs that customers can use directly or a UI which builds on top of those APIs.
1. Service ping information should be kept about which types and how many endpoints are in use.
1. Access to these settings should use the same permission requirements as the existing streaming audit event settings (currently `Owner`)
See additional, specific requirements in each of the child issues and epics.
<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->
*This page may contain information related to upcoming products, features and functionality.
It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes.
Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.*
<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->
epic