archive: add tmt discover/, rpminspect workdir and git clone excludes, drop the dead ones

Follows up on 9f148980, which added --exclude=plans/**/tree/ to keep tmt test checkouts out of the artifact upload. Three more sources of bulk go the same way, and the change is replicated to every environment that loads the archive module.

rsync-options excludes

1a90c69c adds two excludes to all six archive configs:

  • plans/**/discover/ - the tmt discover step data, which the results do not need.
  • rpminspect/tree/workdir - a failed rpminspect request leaves this behind with thousands of files in it.

The comment above the option becomes a list, so each exclude states its own reason.

314827c9 reworks both after Milos's point on the ticket, that discover/ holds tests.yaml and it records what tmt actually discovered for testing. The patterns now anchor on work-*/ rather than plans/: tmt joins the plan's fmf name straight under the work directory, so plans/ only exists for plans named /plans/..., and the tree for /rpminspect sits at work-rpminspect<hash>/rpminspect/tree/. That also covers the rpminspect workdir, so its special case is gone.

The discover step now goes wholesale with tests.yaml kept back:

--include=work-*/**/discover/tests.yaml,--exclude=work-*/**/discover/**

The --include has to come first, rsync applies the first matching rule. And the exclude names the contents, discover/**, not the directory: rsync prunes a matching directory without descending into it, so --exclude=work-*/**/discover/ would leave the include unreachable and tests.yaml would disappear. step.yaml is excluded too, tmt writes one per step per plan.

Measured on a real tmt-1.76.0 run replayed under all three plan layouts, using the option string from each of the six configs: 256 files down to 75, nothing left under tree/ or discover/, and discover/tests.yaml present in every plan.

Environments covered: production/worker-redhat, production/worker-public, dev/worker-local, staging/worker-local, and the dev/worker-public and staging/worker-public templates. The two templates gain only the excludes and keep the rest of their option set as is.

The git clone directory

60ff4eb6 adds a destroy exclude to all six source-destination-map.yaml files. The stage archives ./*, which picks up the test repository clone that test-schedule-tmt creates in the pipeline root under a git-<branch-or-ref> prefix. That clone duplicates the test source we already keep out of the tmt work directories.

Getting the pattern right takes two details from the archive module:

  1. It matches excludes against the raw glob result, and glob("./*") keeps the ./ prefix, so ^git- never fires. The pattern is ^\./git-.
  2. A double-quoted YAML scalar rejects "^\./git-" with found unknown escape character '.', so the backslash is doubled.

The dead citool-debug excludes

b9274962 drops ^citool-debug.txt$ and ^citool-debug.verbose.txt$ from the same destroy entry, in all six files. They miss for the ./ reason above, so ./* has been uploading both debug logs all along.

Removing them changes nothing about what reaches the artifact host. The exclude list just stops advertising a filter it never applied, and what is left is the one pattern that works:

destroy:
  - source: "./*"
    exclude:
      # tmt test source clone, `git-<ref>` in the pipeline root. Exclude patterns are
      # matched against the full `./*` glob result, hence the `./` prefix.
      - "^\\./git-"
    permissions: "777"

The two explicit entries at the end of destroy still upload the debug logs. Their comment about going last so user artifacts stay unaffected has never held, because ./* gets there first. Restoring that ordering needs working excludes plus a check that nothing depends on the current timing, so it stays out of this MR.

Resolves TFT-3594

Assisted-by: Claude Code

Edited by Miroslav Vadkerti

Merge request reports

Loading
Loading