Require consistent state between remotes
With the current state of our release process we have the split between sources of truths for different types of release workflows. Development and build environments are separate.
When we decide to create a new build, release-tools will fetch all the repos that we want to tag with a certain version and push those tags to the build environments. The challenge with this approach is that we assume that the build environment is in sync with the development environments and the versions are tagged on the HEAD of whatever is in the branches in the build environments.
This can lead to situations where development environments show us what we expected to see, but the build environment creates a release with a different content.
We should ensure that we have a consistent state between the two environments.
We can the release-tools to tag versions on a specific commit that we fetch from development environments and have that synced to the build environments. If the commit does not exist, we should have the whole pipeline fail and notify us that we have inconsistent state between two environments.