docs(specs): add S06 storage layer spec
Summary
Adds the S06 storage layer spec and aligns S12 (Container/OCI Local) with the interfaces it defines.
S06 (docs/specs/S06-storage-layer.md — new):
Defines the format-agnostic storage subsystem in two layers:
- Lower layer:
PathResolver(domain identifiers → storage paths),StorageDriverinterface (S3/GCS),FileWriter,Movesemantics, storage middlewares (CloudFront CDN, Google CDN, Redis URL cache). - Upper layer:
BlobStoreservice (StoreBlob,OpenBlob,BlobInfo,DeleteBlob,NewSession,ResumeSession,GetSessionStatus),Sessionlifecycle (chunked upload with incremental SHA-256, two-phase commit),BlobContent(redirect vs stream). - Schema:
blob_storage_blobs(content-addressable, namespace-scoped dedup, hash-partitioned) andupload_sessions(binary existence model, hash-partitioned). - Configuration:
StorageConfigproto message —oneofS3/GCS backend, optional CDN middleware, optional Redis URL cache. - Merges the previously planned S06 (blob storage) and S07 (blob uploads/downloads) into a single spec — the two layers cannot function independently.
S12 updates (docs/specs/S12-container-oci-local.md):
Aligns S12 with S06 interfaces — replaces direct DB/storage references with BlobStore/Session API calls:
- Blob pull uses
BlobStore.OpenBlob(redirect or stream) instead of joining toblob_storage_blobsforobject_storage_key. - Manifest pull uses
BlobStore.OpenBlob/BlobInfoinstead of direct joins. - Blob upload session initiation calls
BlobStore.NewSession, chunk append usesSession.Write/Close, completion usesSession.Commit. - Upload cancellation calls
session.Cancelinstead of directDELETE FROM upload_sessions. - Manifest push step 6a uses
BlobStore.StoreBlob(stores payload + createsblob_storage_blobsrow in one call); handler transaction no longer inserts intoblob_storage_blobs. - Referrers endpoint fetches manifest sizes via
BlobStore.BlobInfoinstead of a join. - Removes
object_storage_keyandcontent_typecolumns fromblob_storage_blobsschema (S06/ADR-007 amendment — content type belongs in format-specific tables; storage paths are computed byPathResolver). - Simplifies
upload_sessionsexpires_atindex to non-composite (purger scans globally, not per-namespace). - Clarifies binary existence model:
Session.Commitdeletes the session row; format-specific rows are created by the handler in a separate transaction. - OQ-12 resolved as superseded by S06.
Other changes:
docs/specs/README.md: Replaces S06 + S07 entries with single S06 (Storage layer, Draft) entry.lychee.toml: Adds HTTP 301/302 to accepted status codes for the link checker.
Related to #7 (closed)
Edited by Pawel Rozlach