Invalidate rollout status cache when weight is changed
Problem
When a weight of Canary Ingress is changed, it takes a few seconds to refresh the displayed weight on UI due to cached data in backend. We should invalidate the cache as soon as the update request has finished, in order to quickly reflect the new value on UI.
Refrence
@shinya.maeda commented
To give some ideas about what could happen, here are the spec on the backend:
environment.json
polling happens every 3 seconds to refresh the environment index page.- For the first time
environment.json
is poked, backend creates a background process to fetch the rollout status (including canary weight) from Kubernetes. While the actual request is being processed, backend returnsloading
status to frontend.- When the actual request finished, the background process stores the result in our cache server (Redis). Backend keeps returning this result to frontend until the cache is expired. Cache will be expired after one minute.
So given that the polling interval is shorter than the cache expiration interval, there could be a case that the displayed value set by frontend could be rolledback by the cache.
By this reason, I suggest not setting/overriding the displayed value by the user's input. Rather, we can simply rely on the polling to reload the actual value on Kubernetes, the cache will soon be expired and the new value will be surfaced. We can show a note in the modal dialog, something like "The weight change request might take a few seconds. The new weight will be reflected on the deploy board shortly." to give users ideas that the weight reflection is a bit delayed. I believe this is enough for now, as Canary Ingress is available for only early adopters today. It won't be available by default until GitLab 14.0.
@afontaine commented.
@shinya.maeda I think your solution makes the most sense then.
I wonder if we could adjust the API such that hitting it would directly invalidate the cache?
I can add a note to the confirmation modal though.
How does that sound @icamacho?