Flaky test: TestDrainTrigger_ScopeDrainedAtTheNormalCadenceKeepsItsHash reads repo-scoped counters as [0 0] after a tick's chunks are enqueued

What fails

TestDrainTrigger_ScopeDrainedAtTheNormalCadenceKeepsItsHash/repo_scoped, in internal/accounting/drain_trigger_integration_test.go, fails non-deterministically on one Postgres version at a time.

drain_trigger_integration_test.go:1247, reached from :586 and :590
Error:    Condition never satisfied
          expected: [2]int64{3, 500}
          actual  : [2]int64{0, 0}
Messages: the repo scoped counters never reached [3 500] after the tick's chunks were enqueued

The counters read [0 0] rather than a partial value, so the work does not appear to run at all inside the wait rather than running slowly.

Evidence that it is non-deterministic rather than a defect

First seen on merge request !1871 (merged), pipeline 2783561754, job 16055729311, on commit 4aed91dd.

Check Result
test:integration: [POSTGRES, 18] failed
test:integration: [POSTGRES, 17] and [POSTGRES, 16] both passed, on the same commit
The same job on the previous pipeline 2783313393, commit 6cc6dac26 passed
The target branch's own latest pipeline 2782905427 success
Does !1871 (merged) touch internal/accounting? No. Its 23 changed files include none, and the test file last moved in a570c673f

Two of three matrix entries running the same test on the same commit and passing is what makes this non-determinism rather than a code fault. The subtest ran 62.03s before giving up.

Relationship to issue #817 (closed)

Issue #817 (closed) tracks TestUploadEmit_ArtifactNewToRepositoryAndNamespace_MovesAllFourCounters, which reads namespace_statistics as 0 after a clean drain. These are different tests and neither should be closed as a duplicate of the other. They are recorded together because the shape is the same and a shared root cause is plausible but unestablished: both are internal/accounting integration tests, both wait for a drain to move a counter, and both read zero rather than a partial value. Issue #817 (closed) was seen on Postgres 17 and this one on Postgres 18, so the version is not the discriminator either.

Whoever investigates one should read the other first.

Condition and event: what a later reader will find

While no fix has landed, a red test:integration job naming this test on a branch that does not touch internal/accounting is this flake, and a retry of that single job is the response. Once a fix lands, the same red is a regression and needs the trace read rather than a retry.

The two counters the assertion names are the repository-scoped pair, so a fix that changes the drain's fan-out or its per-scope claim will move this test's timing and should re-run it deliberately rather than rely on the matrix.

What has not been established

  • The root cause. Nothing here distinguishes a slow drain worker from a job that is never picked up.
  • Whether it reproduces locally. It was not run locally against this branch: the file is behind //go:build integration, and the run that cleared this branch's other obligation was go test -short ./..., which does not compile it. A clean internal/accounting line in that run is therefore not evidence about this test, and should not be read as any.
  • Frequency. One occurrence is recorded here.

This is a bot message 🤖 — /smurfit