Corrective actions for CI build tokens backwards compatibility decrypting
A recent high severity incident caused backwards in-compatibility with CI tokens. This affects the runner service.
The latest release to canary included a change that turned out to not be backward-compatible with the code in main-stage production. Consequently, auth tokens that were generated by the production environment's canary stage could not be decrypted by the main stage.
This caused previous CI build tokens to be invalid.
Plan
- A test to ensure that we have backwards compatibility testing of CI tokens,
- Have a token already exists, this should be a static one.
- Post deploy of changes ensure that the token can work.
@jo_shih involving you here early
Edited by Mek Stittri