archive: add tmt discover/, rpminspect workdir and git clone excludes, drop the dead ones
Follows up on 9f148980, which added --exclude=plans/**/tree/ to keep tmt test
checkouts out of the artifact upload. Three more sources of bulk go the same way,
and the change is replicated to every environment that loads the archive module.
rsync-options excludes
1a90c69c adds two excludes to all six archive configs:
plans/**/discover/- the tmt discover step data, which the results do not need.rpminspect/tree/workdir- a failed rpminspect request leaves this behind with thousands of files in it.
The comment above the option becomes a list, so each exclude states its own reason.
314827c9 reworks both after Milos's point on the ticket, that discover/ holds
tests.yaml and it records what tmt actually discovered for testing. The patterns now
anchor on work-*/ rather than plans/: tmt joins the plan's fmf name straight under
the work directory, so plans/ only exists for plans named /plans/..., and the tree
for /rpminspect sits at work-rpminspect<hash>/rpminspect/tree/. That also covers
the rpminspect workdir, so its special case is gone.
The discover step now goes wholesale with tests.yaml kept back:
--include=work-*/**/discover/tests.yaml,--exclude=work-*/**/discover/**The --include has to come first, rsync applies the first matching rule. And the
exclude names the contents, discover/**, not the directory: rsync prunes a matching
directory without descending into it, so --exclude=work-*/**/discover/ would leave
the include unreachable and tests.yaml would disappear. step.yaml is excluded too,
tmt writes one per step per plan.
Measured on a real tmt-1.76.0 run replayed under all three plan layouts, using the
option string from each of the six configs: 256 files down to 75, nothing left under
tree/ or discover/, and discover/tests.yaml present in every plan.
Environments covered: production/worker-redhat, production/worker-public,
dev/worker-local, staging/worker-local, and the dev/worker-public and
staging/worker-public templates. The two templates gain only the excludes and
keep the rest of their option set as is.
The git clone directory
60ff4eb6 adds a destroy exclude to all six source-destination-map.yaml files.
The stage archives ./*, which picks up the test repository clone that
test-schedule-tmt creates in the pipeline root under a git-<branch-or-ref>
prefix. That clone duplicates the test source we already keep out of the tmt work
directories.
Getting the pattern right takes two details from the archive module:
- It matches excludes against the raw glob result, and
glob("./*")keeps the./prefix, so^git-never fires. The pattern is^\./git-. - A double-quoted YAML scalar rejects
"^\./git-"withfound unknown escape character '.', so the backslash is doubled.
The dead citool-debug excludes
b9274962 drops ^citool-debug.txt$ and ^citool-debug.verbose.txt$ from the same
destroy entry, in all six files. They miss for the ./ reason above, so ./* has
been uploading both debug logs all along.
Removing them changes nothing about what reaches the artifact host. The exclude list just stops advertising a filter it never applied, and what is left is the one pattern that works:
destroy:
- source: "./*"
exclude:
# tmt test source clone, `git-<ref>` in the pipeline root. Exclude patterns are
# matched against the full `./*` glob result, hence the `./` prefix.
- "^\\./git-"
permissions: "777"The two explicit entries at the end of destroy still upload the debug logs. Their
comment about going last so user artifacts stay unaffected has never held, because
./* gets there first. Restoring that ordering needs working excludes plus a check
that nothing depends on the current timing, so it stays out of this MR.
Resolves TFT-3594
Assisted-by: Claude Code