While working on !3952 (closed) I realized a few places in our CI configuration that can use a cleanup.