Ensure our release package will always release on tag

Regardless of flaky qa tests

For now this is fine because we are manually tagging. But when we switch to automatically tagging, we may want to revisit this.