fix(storage): canonical cleanup and unprefixed storage paths (S06 Step 21)

Pawel Rozlachrequested to merge
prozlach/storage-layer-step-21-impl into main

What

S06 plan Step 21 (#166 (closed)), implementation half: the production-behaviour changes and the real-driver conformance test instrumentation. The former Step 21 MR is split so the code lands before its CI wiring — the conformance CI jobs follow in the stacked MR on top of this one.

Runs the upper-layer conformance suite (internal/storage/testsuites) end-to-end against the real pgBlobStore composed with a real StorageDriver, and lands the behaviour fixes that exercising the real stack surfaced.

Behaviour fixes

  • Canonical cleanup on terminal Write/Close faults. The S06 spec ("Canonical cleanup") mandates FileWriter.Cancel + delete the upload_sessions row + mark terminated on the terminal Write (A1a) and Close (G1a) paths, so a later ResumeSession returns ErrSessionNotFound. The stub and the conformance suite already encode this; the real pgSession only set terminated and left the row. (The Commit terminal paths already ran it.) The two Step-8 mechanism tests that pinned the buggy post-state are updated.
  • Unprefixed storage paths across the stack. The S3/GCS drivers required a leading slash (Container Registry contract) while the spec and PathResolver produce unprefixed paths. Tighten the path regexp, normalise the S3 root prefix once at construction, harden path validation/joining, and invert the driver testsuite's valid/invalid path sets. Root config stays at both the driver (RootDirectory) and PathResolver levels; final SDK keys stay unprefixed.

Test instrumentation

  • A real-driver Factory/ClockedFactory (env-selected backend; PostgreSQL from the package TestMain via DSN or testcontainers) and the conformance/fault suites (realdriver_*_integration_test.go).
  • Inert gofail seams in the real upper layer (pg_session.go, pg_blobstore.go): writeStagingFail, closeFlushFail, commitFlushFail/commitFinalizeFail, commitMoveFail, commitTxFail/commitTxFailRemoveBlob, commitRowVanished, commitZeroLengthDivergence, resumeOffsetDivergence. They activate only under go tool gofail enable internal/storage; the fault tests skip when not compiled in. The fault suite is sequential (process-global failpoints), the happy-path suite parallel, so armed seams never overlap a concurrent run.
  • The B1 shared-staging race is skipped for S3 only: S3 multipart staging completes the shared upload, so the loser hits NoSuchUpload before the CAS; GCS reaches the dirty CAS and passes (confirmed against conformance:gcs-key-creds).

Verification

  • Default build/vet clean; the gofail-enabled test binary compiles and the round-trip leaves no *.fail.go.
  • The updated mechanism tests pass against testcontainers PostgreSQL locally.
  • End-to-end fault behaviour is validated in CI by the conformance jobs in the stacked CI MR; conformance:s3-seaweedfs needs no real-cloud infra.

Dependencies

  • Stacked on !580 (merged) (work-item-182 real-backend driver jobs): targets the !580 branch; retarget to main once !580 (merged) merges.
  • The CI jobs that run this instrumentation are in the stacked MR on top of this one (prozlach/storage-layer-step-21 → this branch).

Related to #166 (closed)

🤖 Generated with Claude Code

Merge request reports