ci: Only test component when files changed

This reduces CI time a lot, since the component tests are by far the most time consuming jobs (since they have to run docker and build a container).