[BB-856] Use locks when updating Consul.
Test instructions:
- Go to https://stage.console.opencraft.com/instance/3360/edx-appserver/1064/
- SSH into haproxy-stage-1.net.opencraft.hosting and haproxy-stage-2.net.opencraft.hosting
- On one server run
consul watch -type=keyprefix -prefix=lock/ consul kv get -recurse -detailed lock/
- On the other server run
consul watch -type=keyprefix -prefix=ocim/instances/362/ consul kv get -recurse ocim/instances/362
- Deactivate one server, activate another
- After some time passes, check the spam from the watches - the lock is being worked on by both Ocim and consul-template across the HAProxy servers.
- After it's settled down, check
consul kv export lock/
and notice no lock is taken anymore by anyone.