Simplify GitLab architecture by shipping Minio

As part of simplifying GitLab architecture, and implementation, we should consider including Minio as our storage backend for every type of access that is based on filesystem.

Doing that, would allow us to remove a lot of boilerplate for manually accessing files and hide that behind abstraction for filesystem access.

I could see the following transition period:

  1. We include minio in GitLab stack,
  2. We configure minio to expose bucket for each storage space that is present today,
  3. We use single ACCESS/KEY pair for all these buckets,
  4. On each node, we run the extra Minio instance,
  5. We automatically configure Rails to use Minio if object storage is not used,
  6. We continue to support local storage until we can get rid of the transition period,
  7. We remove code to handle multiple storage buckets, fallback to a single one,

Open matters:

  1. How we support different storage buckets? (today we allow to have local and remote storage),
  2. Can we remove support for multiple storage buckets?
  3. Can we drop support for that with %12.0?
  4. Can we say that we only allow minio-based or S3-configured based access?

Why this is needed?

  1. We can remove a lot of complexity from GitLab Rails,
  2. We can disallow access to local files, and thus prevent a lot of file system accesses,
  3. Today, Minio can be hidden behind Workhorse and not exposed outside, us Workhorse does support proxying access,
  4. We can adapt all tools to have only one storage interface: remote-based, which greatly simplifies the architecture.
  5. Doing that with Minio, makes our application to be Cloud-Native (by default), not by the particular implementation,

Items to consider

Edited by silv