Maven Virtual Registry: Database models

🔭 Issues plan

🗒 Description

This issue will deal with all the new database tables and models that we need for the Maven Virtual Registry. Being the first package format to be implemented, we have a few tables to introduce:

classDiagram
    class R["VirtualRegistries::Parent"]
    R : Pkey id
    R : Fkey group_id

    class Reg["VirtualRegistries::Packages::Maven::Registry"]
    Reg : Pkey id
    Reg : Fkey parent_id
    Reg : Integer cache_validity_period_hours

    class RegU["VirtualRegistries::Packages::Maven::RegistryUpstream"]
    RegU: Fkey registry_id
    RegU: Fkey upstream_id

    class U["VirtualRegistries::Packages::Maven::Upstream"]
    U : Pkey id
    U : String url
    U : EncryptedJson credentials

    class CR["VirtualRegistries::Packages::Maven::CachedResponse"]
    CR : Pkey id
    CR : Fkey upstream_id
    CR : String relative_path
    CR : object storage related fields
    CR : String upstream_etag
    CR : Timestamp upstream_etag_checked_at
    CR : String content_type
    CR : SmallInt status
    CR : Timestamp last_downloaded_at
    CR : Integer downloaded_count

    R "1" --> "1" Reg
    Reg "1" --> "1" RegU
    RegU "1" --> "1" U
    U "1" --> "0..*" CR

Details

VirtualRegistries::Parent: holds the reference of the parent object. In this case and first iteration, a root Group:

id: Primary key as usual.
group_id: Foreign key on a Group. Model validation: this must be a root Group.
Notes:
- Not very useful in the first iteration but required so that the entire feature can be "anchored" at a different existing level (such as Project or Organization).

VirtualRegistries::Packages::Maven::Registry: the registry object for the Maven package format:

id: Primary key as usual.
parent_id: Foreign key on a VirtualRegistries::Parent. Restriction for the first iteration: unique .
cache_validity_period_hours: Integer. Period (in hours) that the backend should consider cache objects as valid (and thus, not check them against the related upstream).
Model validations:
- Uniqueness on parent_id (first iteration only: only one maven registry per parent).

VirtualRegistries::Packages::Maven::RegistryUpstream: the bridge between a Maven Registry and a Maven upstream.

registry_id: Foreign key on a VirtualRegistries::Packages::Maven::Registry.
upstream_id: Foreign key on a VirtualRegistries::Packages::Maven::Upstream.
Model validations:
- Uniqueness on upstream_id (first iteration only: only one maven upstream per registry).
Notes:
- Not very useful in the first iteration but required when we go multiple upstreams + shareable upstreams between registries.

VirtualRegistries::Packages::Maven::Upstream: holds the data on how to contact the upstream, mainly the url and credentials if any to use.

id: Primary key as usual.
url: String. Url format. Required.
credentials: Encrypted field storing a json structure. The model should expose the fields that are available: username, password.
Model validations:
- username and password need both to be set or empty.
- url is required and should be validated to have an URL like format.

VirtualRegistries::Packages::Maven::CachedResponse: holds a response from an upstream. Main fields are the relative path and the columns dedicated to the object storage file.

id: Primary key as usual.
upstream_id: Foreign key on a VirtualRegistries::Packages::Maven::Upstream.
relative_path: Text. Required.
file: text. Object storage related field. Required.
file_store integer. Object storage related field. Required.
size: integer. Object storage related field. Required.
object_storage_key: Object storage related field. Required.
upstream_etag: text. The etag sent by the upstream (could be blank).
upstream_checked_at: timestamp. Last time that we checked upstream. Required.
content_type: Text. Content type value as sent by the upstream. Required.
downloaded_at: Timestamp. Last time that this cached response was downloaded.
downloads_count: (positive) Integer. Amount of times that this cached response was downloaded.
Model validations:
- We have a few fields required (see above).
- Uniqueness on [upstream_id, relative_path] (only one relative_path per upstream)

Notes:

VirtualRegistries is a new bounded context under category virtual_registry.
We have some tables that are not useful but these indirections will be required in follow ups.

⏫ Update

During the implementation, we discovered this requirement:

All newly created cell-local tables are required to have a sharding_key defined in the corresponding db/docs/ file for that table.

and this challenged the existence of the Parent table. Thus, we are not implementing it for the time being:

classDiagram
    class Reg["VirtualRegistries::Packages::Maven::Registry"]
    Reg : Pkey id
    Reg : Fkey parent_id
    Reg : Integer cache_validity_period_hours
    Reg : Fkey group_id

    class RegU["VirtualRegistries::Packages::Maven::RegistryUpstream"]
    RegU: Fkey registry_id
    RegU: Fkey upstream_id
    RegU: Fkey group_id

    class U["VirtualRegistries::Packages::Maven::Upstream"]
    U : Pkey id
    U : String url
    U : EncryptedJson credentials
    U : Fkey group_id

    class CR["VirtualRegistries::Packages::Maven::CachedResponse"]
    CR : Pkey id
    CR : Fkey upstream_id
    CR : String relative_path
    CR : object storage related fields
    CR : String upstream_etag
    CR : Timestamp upstream_etag_checked_at
    CR : String content_type
    CR : SmallInt status
    CR : Timestamp last_downloaded_at
    CR : Integer downloaded_count
    CR : Fkey group_id

    Reg "1" --> "1" RegU
    RegU "1" --> "1" U
    U "1" --> "0..*" CR

Same constraints apply.

Edited Jun 21, 2024 by David Fernandez