feat(storage): real Session lifecycle + commit protocol (S06 Step 9)

What

Implements the real PostgreSQL Session lifecycle and commit protocol, per S06 Step 9 of docs/plans/2026-05-15-storage-layer.md. Final real-implementation MR in Stage 2 Track A; completes the Session interface declared in Step 1 and re-runs the full Stage 1 conformance suite (Steps 2–4) against the real impl.

Adds, on top of Step 8's chunked upload + deferred persistence:

  • ResumeSession — restores a session from upload_sessions with no row lock; opens Writer(stagingPath, offset=size_bytes) and, on a non-live upload, probes Exists to pick a recovery branch (normal / finalize-only D1b / DB-finalize-only D2a / unrecoverable D2b). Backend-ahead-of-size_bytes divergence is terminal (*OffsetDivergenceError).
  • GetSessionStatus — lightweight read, no Writer, no recovery.
  • Session.Commit — the commit protocol: hash finalize → digest verify (skipped on empty expectedDigest, the Maven path) → zero-length assertion → FileWriter.Commit()PathResolver.ObjectPathMove() (with BlobUserMetadata → backend headers, or dest-exists shortcut with metadata elided, first-write-wins) → atomic INSERT blob_storage_blobs + DELETE upload_sessions in one transaction.
  • Session.Cancel — staging + row teardown (best-effort storage delete).
  • Crash-recovery model: size_bytes CAS + dirty poison-pill reconcile concurrent resumes without locking (S06 "Consistency & Crash-Recovery Model"); GC grace period enforced via blob_storage_blobs.created_at.

Notes for reviewers

  • Derived public-API seam, not in the plan's Naming Conventions table. This step adds a variadic NewBlobStore plus storage.WithClock / storage.WithSessionTTL — the real-impl analogue of the in-tree stub.WithClock / stub.WithSessionTTL, required by the plan-mandated RunSessionExpiry ClockedFactory re-run. Backward-compatible. Flagging for a possible plan-amendment to enumerate them.
  • Fault-injection mechanism. D1a/D1b/D2a/D2b and offset-divergence are driven through one-shot mock-driver knobs + direct DB state (not gofail seams), since test:storage-failpoints does not cover internal/storage and these run under //go:build integration. The plan sanctions "equivalent driver/DB faults for Step 9's PostgreSQL implementation"; driver-side gofail seams remain Step 16/17 scope.
  • Fixture correction (commit 2). The initial ZeroLengthViolation fixture reset only size_bytes, which tripped the resume-time *OffsetDivergenceError guard before the Commit-time zero-length assertion was reachable — mutually unsatisfiable with TestPgSession_Resume_OffsetDivergenceRejected. Resolved per the spec's terminate-on-divergence model (no self-heal): the fixture now drops backend staging to 0 alongside the size_bytes reset so the desync is reached cleanly. Implementation unchanged.

Database: query plans (S06 Step 9 commit-path queries)

EXPLAIN (ANALYZE, BUFFERS) against a schema built from structure.sql (64 hash partitions per table), seeded with 200,000 upload_sessions and 500,000 blob_storage_blobs rows, then ANALYZEd. Every commit-path query is a partition-pruned index scan — no sequential or bitmap heap scans.

Query (call site) Plan Index Exec
reReadDirty SELECT Index Scan (1 partition) …_namespace_id_upload_id_idx 0.26 ms
activeSessionRow SELECT (Resume/GetStatus) Index Scan; repository_id+expires_at residual filter …_namespace_id_upload_id_idx 0.015 ms
size_bytes CAS UPDATE (Close/Commit 5a) Index Scan …_namespace_id_upload_id_idx <1 ms
flagDirty UPDATE Index Scan …_namespace_id_upload_id_idx 0.08 ms
commitDBTx DELETE session Index Scan …_namespace_id_upload_id_idx 0.02 ms
blobLocation SELECT Index Scan …_namespace_id_sha256_idx 0.03 ms
commitDBTx INSERT … ON CONFLICT DO NOTHING Conflict arbiter index unique_blob_storage_blobs_on_namespace_id_and_sha256 0.31 ms

Representative plan (reReadDirty):

Limit  (cost=0.28..8.30 rows=1 width=1) (actual time=0.247..0.248 rows=1 loops=1)
  ->  Index Scan using upload_sessions_p00_namespace_id_upload_id_idx on upload_sessions_p00
        Index Cond: ((namespace_id = '…'::uuid) AND (upload_id = '…'::uuid))

Conclusion: all Step-9 commit-path queries are served by the existing unique indexes on (namespace_id, upload_id) and (namespace_id, sha256), with hash-partition pruning to a single partition. No new index required. (go-jet sends these as $1-parameterized statements; the literal-value plans are the equivalent custom plans.)

Related to #158 (closed)

Edited by Pawel Rozlach

Merge request reports

Loading
Loading