Zoekt-indexer tasks API doesn't respect graceful shutdown logic
Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.
Summary
As part of #477194 (closed), we restarted one of the pods. In the logs I've noticed that even after the Gracefully shutting down... message, we still processed new requests.
Before the tasks API, we relied on the API server to stop receiving new indexing requests. Now, the ticker initiates the requests even during the shutdown phase.
Steps to reproduce
Create zoekt tasks. Press <Ctrl>-C, you will see that new IndexRepository start and TaskRequest ticker information messages in the logs.
What is the current bug behavior?
We process new indexing requests after shutdown is initiated https://gitlab.com/gitlab-org/gitlab-zoekt-indexer/-/blob/122ff11d46fe28cbf69a5c299d4323482cebdaad/cmd/gitlab-zoekt-indexer/main.go#L199
What is the expected correct behavior?
This shouldn't happen
Relevant logs and/or screenshots
Possible fixes
I believe that we should stop the ticker as soon as we received the shutdown command.
Note: We also need to check that indexing operations started by the internal API are stopped gracefully.