CI: macos_server fails some tests

In !976 (merged) we are adding a macOS CI job that brings up the server, helps us fix up our docs etc.

Sadly it doesn't pass the tests. Perhaps/likely related to #2194?

Perhaps someone who runs mac locally can help us debug these things.