Remove default config

Installing pre-commit requirements doesn't take that long and keeping the default up-to-date in the image is not worth it (and not really possible).

Bump pre-commit version at the same time.