Add Gradle build cache optimizations

This adds some small optimizations to improve build speed, although it does not improve a lot. After digging quite deep, the main issue is with the dependencies between the projects, and the bigger improvements will stem from decoupling and maybe partially rewriting. I have some changes almost ready, but those will go into a different MR.