docs(specs): the per-format slices discard the plaintext, not backfill

What

Five merged spec lines prescribed a backfill from the interim tmp_plaintext_* credential columns into the encrypted_* ones. No such backfill exists or is planned, so the specs contradicted what the per-format slices actually do: they discard the plaintext values and report the repositories that held credentials, so an operator can re-enter them.

No SQL migration can encrypt a column, because that needs the Go crypto stack and the root key, and no Go-side pass owns writing one. The re-encrypt operator command already resolves the root key and walks credential tables, but no phase of it encrypts a plaintext column today.

The branch then records what Maven's read/write move actually changed, and what it left alone. maven_remote_repositories carries the ADR-007 encrypted_* unit and its reads and writes go through it, while its interim tmp_plaintext_* columns and their own CHECK stay declared until the drop step lands.

Merge order

Step 4's first part merged on 2026-09-11 as eb965a43e, so the plan's "merged before Step 4 merges" ordering was not met and the specs have been disagreeing with the code since. This MR closes that gap; nothing gates on it merging first any more.

The branch is rebased onto 85530efe0, which is what makes the column-encryption.md reference below resolve in this branch's own tree rather than only after a merge.

Nine open MRs touch a file this one touches, re-derived at head aeae1107c. Eight of them target main, and seven of those eight merge cleanly into this branch, checked with git merge-tree --write-tree: !2642 on S04-b, !2641 and !2514 on S16, !2629 on column-encryption.md, !2623 on S13, !2563 on S04-a, and !2533 on S17. No merge order is required between them and this MR. !2625, which an earlier round listed, merged on 2026-09-14 and has left the open set.

Two conflict against this branch, and the pair is not the pair the previous round named. !2627 (Step 10 part 3/3) has been retargeted to main since that round, and conflicts on column-encryption.md. !2630 (Step 5 part 2/2) is now the only one still targeting a sibling branch, so the merge measured above is not the merge it performs; it conflicts on column-encryption.md, S04-a, S04-c, S13, S14 and S16. Whichever of the two reaches main second rebases onto the other; neither gates this MR.

Review round: the same fact at the sites the first pass missed

The first pass narrowed the plaintext claim where the columns are named and left it stated unconditionally elsewhere. docs: narrow the plaintext-credential claim to the sites that hold it closes that:

  • S13 had four more passages saying every format stores credentials in plaintext: the overview, the credential-management lead-in, the config-secrets note, and the lead-in to the interim-column table itself, which had lost its "Until encryption lands" qualifier without gaining a scope and so contradicted the paragraph four lines above it. The crypto-shred follow-up no longer says "when encryption lands" either, since that is now per table.
  • S17 carried the fact twice, in its credential security note and in the S07 reconciliation follow-up.
  • S16 said both things at once. Its data model had the encrypted_* names "return once #417 (closed) lands" while its own follow-up said the table already carries them, and its column table, which this spec calls the authoritative source a migration is transcribed from, listed neither the ciphertext pair nor the key-wrap trio that 20260904140000 added. Both are in that table now, mirroring the rows S14 already carries. ADR-007 publishes all five for this table, so they add no divergence to the list S16's follow-up tracks. The #417 scope list now says which of its four parts each table has reached.
  • S04-C overstated the refusal: a Maven credential write is refused without the block only when it supplies a credential. A clear, and the clear a url change carries, write NULL to the whole unit and encrypt nothing, which is the operator's route back on a row whose read fails closed. S13's own fail-closed table already said so, and column-encryption.md carried the same loose wording, so both sites are fixed.
  • S14's encrypted-unit CHECK now names its five columns instead of counting rows back, which reached tmp_plaintext_password.
  • S13's drop gate now carries the condition column-encryption.md states: the error line reads as a gate signal only from an environment whose migrations are applied, because the report takes no dependency on migration readiness.
  • logging.md gains the boot line's keys, levels and bounds. Two specs now gate a plaintext-column drop on reading that line, and an absent record is the case that fails the gate, so it belongs in the catalog. format is documented as maven today, since only Maven's table registers a report.

Review round: the sites the second pass missed

Eleven blocking findings, each landing as its own commit:

  • S04-A hedged the all-NULL VALIDATE CONSTRAINT guarantee against a migration that also populates the encrypted unit, and attributed that open branch to S13. S13 rules it out, so the guarantee is now unconditional for the add-only shape these migrations take. Its interim-constraint rationale also still called the plaintext pair the live credential store; that round replaced the claim with the pair staying declared under its own CHECK until the drop. The round below then took the per-format status out of the same block altogether, for the reason its last entry gives.
  • S15 joined the file set, for the reason above.
  • The discard premise was stated without its backing measurement. column-encryption.md now carries the reading (2026-09-03: no traffic on .com, test traffic only on staging) and the obligation on the slice that moves a table to re-confirm it against the environments its move reaches; S13 states the same beside its discard bullet, and S16 and S15 point at column-encryption.md rather than restating the premise bare.
  • S04-C said "the remaining per-format encrypted_* steps", which misnames what is left: all three tables already carry those columns, so what remains per format is the credential read/write move and then the drop.
  • S17 read as if Maven's credential work were finished, in the credential security note and again in the S07 reconciliation follow-up. Both now carry the leftover-row qualifier.
  • S13's crypto-shred re-enable bullet read per-table, so a namespace re-enabled before Container's or npm's move looked like it would keep those tables' plaintext credentials. It now states that the re-enable clears every remote-repository table's credential material, and separates what the composite FK actually forces: a row still on its tmp_plaintext_* columns carries NULL in ns_key_id.
  • S13's drop gate gave the pass condition without naming whose line counts; it now says every environment the drop reaches, with the reading recorded in the drop MR's description, matching S14 and the plan.
  • S13 attributed the discarded-credential state to a migration; it now names the move that discards the values and the drop that makes it permanent.
  • S13's Security Considerations credential bullet still said plaintext at rest during the interim; it now mirrors the per-format split.

Two findings are follow-ups rather than changes here, because step MRs do not edit the plan file. Neither has a carrier yet; see Plan follow-ups still unowned:

  • Step 4's acceptance, echoed in its Dependencies and inherited by Steps 7 and 10, says a credential write is refused with the block absent, where S04-C now qualifies that only a write supplying a credential is refused.
  • Steps 7 and 10 accept only "as Step 4 for Container/npm", and Step 4's acceptance carries no re-confirmation arm, so nothing in the plan obliges the moving MR to re-read the discard measurement. Those two steps need an acceptance arm of their own: the moving MR re-reads the measurement against the environments its move reaches and records the reading in its description.

Review round: the round-two findings on the round-one fixes

Nine blocking findings on the corrections above, eight of them landing as their own commit and one as the plan follow-up recorded below.

  • S14 framed all eight of its divergences as pending an ADR-007 amendment, and S15 said the same of each of its four. The interim credential columns are not in that set, because ADR-007 never publishes those column names, so no amendment can remove them: that divergence closes when a table's reads and writes move onto the encrypted unit and the columns are dropped. Both lists now split the framing. The counts are unchanged at eight and four; S14's seven amendment-bound divergences are the snapshot_metadata_always_revalidate column plus the six hardenings, and S15's three are the kind=2 value, the dropped package_json column and the dropped npm_remote_tags table.
  • S13's re-enable bullet said each table joins the FK-forced half of the composition as its reads and writes move, which reads as if every credential row on a moved table were FK-forced. It is a property of the row: a row carrying an ns_key_id is FK-forced, a row carrying NULL there is not, and the NULL set is an unmoved table's rows plus a moved table's leftover pair-only rows until the drop. Those come out because the transaction names their table.
  • S04-A's re-enable step said leaving credential rows behind is structurally impossible because the ns_key_id FK blocks the key-row delete. That argument reaches only rows carrying a key reference, which is the reassuring answer for exactly the rows that can be missed. The step now scopes the impossibility to those rows and names the naming-only case beside it.
  • S13's credential follow-up and its Resolutions entry still described the work as moving the plaintext columns to the ADR-007 shape and dropping them. All three tables already carry that shape (20260904140000, 20260904160000, 20260904170000) and the values are discarded, not moved. Both now mirror S15 and S16: what remains is Container's and npm's read/write move, then every table's drop. The Resolutions Why is dated to the decision, with a sentence saying the framework has since landed and that per-table adoption is what keeps the resolution alive.
  • The purge's guarantee rested on a hand-named list with no home, where the corpus's other table-join obligations have one. column-encryption.md's consumer-table contract now carries the re-enable delete set beside the two rotation-sweep obligations, says it is independent of both, and says a test is what catches an omission because the composite FK holds only a row carrying an ns_key_id. It also records that the set has no code home yet, since DeleteNamespaceKeys removes the namespace_encryption_keys rows alone, so the obligation reads correctly before and after the delete is built. S13 points at it rather than restating it.
  • S04-C's Dependencies row still said "npm and maven tables implemented (interim plaintext); container planned", which this spec's own absent-block paragraph and S16's #417 scope list both contradict. The State cell now carries the per-format split.
  • S04-A's interim-constraint rationale said the drop is permitted a release after the code stops referencing the columns. No release satisfies that: a moved Maven credential write still writes NULL into the interim pair, the leftover-plaintext line still reads it, and the drop step removes both together with the columns. The rationale now points at the gate that operates, and a new paragraph states it in full: the read/write move released rather than merely merged, plus a zero line in every environment the drop reaches. It says plainly that this is weaker than database-migrations.md's two-release rule read literally. That paragraph's own account of what the drop release does was corrected again in the round below.
  • That same SQL block carried a per-format live-store sentence, which diverges from the merged 20260904170000 migration and would need amending again once Container's move lands. Since the block is the copy-source later readers work from, the rationale is now status-neutral and points at S13's credential columns section and S14's Credential columns paragraph, where the per-format statements already live.

Review round: the round-three findings on the round-two fixes

Three blocking findings on the round-two corrections above, each landing as its own commit. Two are the defect class that round was fixing: a sentence this branch had just edited disagreeing with another passage it had just edited. One of the two also adds to the plan follow-up below.

  • S14's Credential columns paragraph still gave database-migrations.md's two-release rule as the drop's reason, where S04-A's new paragraph says that rule is not the gate that operates. The shipped write path agrees with S04-A: a credential write through the encrypted unit still names the interim pair, in updateMavenRemoteCredentialsStmt and mavenRemoteCredentialUnitAssignments, so nothing on main has stopped referencing the columns yet. !2629 is the code-only release that does, a release ahead of the drop, which is why the round below bounds every spec sentence about the nulling by it. The paragraph now states the gate this spec's own Follow-ups entry already carries and points at S04-A for why the two-release rule is not it.
  • S04-A's new paragraph described the drop as it stood before Step 5 was split in two. Under the split a code-only release stops the binding ahead of the drop, so no pod on the previous image writes a dropped column and the window the paragraph accepted does not arise. What the drop release still crosses is the leftover-plaintext line's own read, which retires with the columns: a pod booting on the previous image counts leftovers against columns that are gone and emits that line's error, the mirror of the pre-migration boot S13 already records for the same line. The paragraph now carries the order and that residual in place of the accepted window.
  • column-encryption.md called the sweep registry "pinned by a test that fails when a credential table is missing from it". TestDefaultOperatorCredentialTables_RegistersMavenOnly asserts the registry's length and Maven's table name, so it reddens on any change to the registry's own contents and stays green on the omission the section exists to prevent: a credential table that exists in the schema and was never registered. The same paragraph's lead-in already said a table left out of a set still compiles and still passes its own tests, so the two disagreed. The registry pin is now named a contents assertion with what it does and does not reach, and the delete set's test is required to derive its expected membership from the schema, every table carrying the key-wrap trio, so an omitted table reddens it rather than inheriting the blind spot.

One more finding is a follow-up rather than a change here, for the same reason as the two above it, and it has no carrier either (Plan follow-ups still unowned):

  • Step 4's Scope says "nothing touches tmp_plaintext_*", which the shipped code contradicts: the credential write still names the pair, and stopping that is Step 5's first part rather than Step 4. Step 4's Scope and Step 5's Files list both need the split's account.

Review round: what the round-three edits made false elsewhere

Two blocking findings, both about what the round-three corrections above made false at other sites rather than about the passages those corrections landed on. One lands as a commit; the other is a correction to this description.

  • S04-A's split paragraph now documents that a code-only release stops the credential write naming the interim pair, and nine spec sentences described the nulling, or defined the leftover set by it, without that boundary. Each turns false the moment that release ships, and no MR carries them: !2629 is the un-bind release and its doc pass touches column-encryption.md, credential_table.md and the runbook but no docs/specs/ path, !2630 rewrites S13 and S14 only at the drop a release later, and S17 is in neither. Every one now names the condition and its event instead of today's state: while the write statements name the pair a credential write nulls it and the leftover set shrinks as credentials are re-entered; once the release stops them naming it the set freezes, because a write leaves any plaintext the row already holds and only SQL, or the parent repository's reap, removes one. S14 :81-82 and :572 drop "shrinking set" and "a row not written since the move" as that set's definition, since a row written after the release can hold a pair too. S17 :1648 adds what the release costs its own surface: re-entering the credential is the caller's remedy only until then.

    The set was re-derived rather than copied from the finding, and it differs twice. S14 :80, which the finding did not list, carries the same fact and is in. S13 :1202 is out, with S04-A :1714 beside it: both characterize the ns_key_id NULL set for the crypto-shred purge, which a credential write populates whether or not it also nulls the pair, so the release does not move them. S04-C :77 names one uncovered case rather than the whole set and stands as written. The same fact in docs/dev/column-encryption.md :140 and :157, internal/datastore/credential_table.md :86 and the maven_remote_credentials_test.go comment is left to !2629, whose doc pass already carries the boundary and which merges clean here today; editing those paragraphs from both branches would only create the conflict that measurement says is absent. The plan's two inheritance paragraphs (:623-626 and :2073 at origin/main) are about Container and npm, whose write arms still name their interim columns, so the release does not falsify them.

  • This description recorded the three plan follow-ups as joining !2633. They are not in it, and no open MR carries them. The section below is the correction.

Plan follow-ups still unowned

Three findings from the rounds above are plan-file edits, which a step MR does not make, so they are recorded rather than fixed:

  • Step 4's acceptance, echoed in its Dependencies and inherited by Steps 7 and 10, says a credential write is refused with the block absent, where S04-C now qualifies that only a write supplying a credential is refused.
  • Steps 7 and 10 accept only "as Step 4 for Container/npm", and Step 4's acceptance carries no re-confirmation arm, so nothing in the plan obliges the moving MR to re-read the discard measurement against the environments its move reaches.
  • Step 4's Scope says "nothing touches tmp_plaintext_*", which the shipped code contradicts: the credential write still names the pair, and stopping that is Step 5's first part rather than Step 4. Step 4's Scope and Step 5's Files list both need the split's account.

No open merge request carries any of the three, re-derived at head aeae1107c against every open MR touching docs/plans/2026-09-03-s04a-per-format-credential-columns.md: !2633 (21 added lines) amends only Step 5's gate-reading ownership, !2631 records Step 10's three merge requests and one Files bullet, and !2543 corrects the plan's stale claims and citations without reaching any of the three. An earlier round of this description said all three would join !2633; that was wrong, and this section replaces the claim. They still need a docs(plans) MR, and while none is open this description is the only record of them, so whoever opens the next docs(plans) MR on this plan takes them.

Lines amended

This branch's own line numbers at head aeae1107c, from a diff against origin/main 85530efe0, which is this branch's merge base.

File Lines
docs/dev/column-encryption.md 11-12, 43-44, 216-236
docs/dev/logging.md 114, 122-140
docs/specs/S04-a-column-level-encryption.md 407-418, 477-511, 557-558, 560-561, 563-566, 1708-1719
docs/specs/S04-b-encryption-key-rotation-tooling.md 455
docs/specs/S04-c-column-level-encryption-wiring.md 31, 73-77, 131-133, 185
docs/specs/S13-virtual-remote-foundation.md 18, 87-90, 92, 101, 103-113, 599, 622-626, 992, 1144-1145, 1196-1201, 1203-1206, 1208, 1232-1234, 1236
docs/specs/S14-maven-remote.md 67-68, 72-82, 94-100, 572-573, 599-600
docs/specs/S15-npm-remote.md 127-135, 139-158, 175-179, 1162-1173
docs/specs/S16-container-remote.md 53-58, 95-100, 1348-1351
docs/specs/S17-rest-management-api.md 1647-1649, 1780-1781

346 reviewable LOC across ten files, 256 added and 90 removed.

The plan's dependency names the five backfill lines as S16:56 and :1258, S04-b:445, and S04-c:128 and :180. Two of its citations went stale rather than the facts they record: the second S16 line is :1349 at this base and the S04-b line is :455.

Two S16 lines still carry the word. :1294 is about the last_health_status CHECK constraint on the npm and Maven tables, a different subject from credential material. :58 and :1349 use it to say the values are discarded rather than backfilled, which is the point of this MR.

docs/specs/S15-npm-remote.md was left untouched in the earlier rounds on the grounds that npm's table has not moved. Its plaintext statements are indeed still accurate, but its column table is the authoritative source a migration is transcribed from, and it listed none of the four columns 20260904160000_add_npm_remote_encrypted_credentials.sql shipped. The third round below gives it the same both-shapes treatment S14 and S16 carry.

What each file says now

  • S14, the authoritative source for the Maven remote tables, lists both shapes: the five encrypted_* and key-wrap columns the add migration landed, and the two interim columns that are still there. The divergence count stays at eight, because the ADR-007 credential-column divergence is still open for this table, and the Follow-ups entry is the drop rather than the migration. Security considerations says the encrypted unit is the live store and that a row not written since the move still holds its plaintext, so the closed-beta-blocking posture holds for those rows.
  • S13 keeps Maven in the interim-column table and in the plaintext CHECK bullet, and separates the read/write move from the drop. Its discard bullet carries the condition docs/dev/column-encryption.md attaches to the discard: it is acceptable while no deployment holds a production credential. The drop gate reads fail-closed, so an absent line or the error line a failed count emits fails it rather than passing it, matching the plan's Step 5 acceptance.
  • S04-C says the Maven table reads its credentials from the encrypted unit alone, so a missing encryption block fails a read on a row carrying that unit closed and refuses a write that supplies a credential, and names the case the block does not cover: a row whose credential predates the move reads as an anonymous upstream instead.
  • S04-A's enc_ infix rationale stays in present tense and adds why the infix survives a drop. Its each is corrected in the same pass: the Maven and Container tables carry an interim plaintext constraint, the npm table carries none, which the spec's own constraint table at :468 already implied by marking npm's CHECK new with no predecessor.
  • S16 carries both column shapes in its data model and its table, says the interim pair is still this format's live store because no write populates the encrypted unit yet, and says the discard is reported rather than silent.
  • S15 now does the same for npm_remote_repositories: the column table lists encrypted_auth_token and the key-wrap trio beside the interim column, the unit is four columns rather than five because npm stores a single bearer token, and the interim column is still named as npm's live store because npm's reads and writes have not moved. Its follow-up is the read/write move and then the drop.

Testing

Documentation only, no code. markdownlint-cli2, lychee with --include-fragments, and vale pass on all ten files. No added line reaches MD013's 800-character cap.

Every claim about current behavior was checked against origin/main rather than against the prose: the five Maven columns and their CHECK, index and composite FK in 20260904170000, the interim pair nulled on both Maven write paths (updateMavenRemoteCredentialsStmt and mavenRemoteCredentialUnitAssignments), the clear arm returning before the encryptor check in setCredentials, Container's read still on TmpPlaintext* with no production writer for its encrypted columns, npm carrying no CHECK on its interim column, and the boot line's three levels and capped sample in wire_remote.go.

The round-three claim about the sweep registry's test was read off the test rather than off either the finding or the prose it corrects: TestDefaultOperatorCredentialTables_RegistersMavenOnly asserts only require.Len(tables, 1) and tables[0].TableName(), and takes no reading of the schema, so nothing in it can observe a credential table that exists and was never registered.

Related to #417 (closed)

Edited by Dzmitry (Dima) Meshcharakou

Merge request reports

Loading
Loading