Leave rugged to be installed from the gemfile. Set build configuration with bundle config.

We had 2 versions of rugged installed. Builds were passing due to caching so we shipped both 0.21 and 0.23 while only 0.23 was really used.