Find a way to adjust the tests to account for permissions-setting issues

When trying to add the CI pipeline, I am getting erroneous failures, presumably because whatever container the CI is running in won't let the code permanently alter the permissions on some files and folders see this pipeline. I have seen something similar to this with Rust playground as well.

Ideally, these tests would not require anything special to be done in order to run. If there's no way around it, we should:

1. set up that special thing for the gitlab CI
2. detect whether we're in an environment that supports running these kinds of tests and make the tests optional, so people don't get annoying local failures because their environment doesn't work as expected.