Implement callback API

Problem to solve

Currently, we rely on blocking requests from Rails to indexer to understand if the task has been completed.

Proposal

We should introduce a new callback action to the Internal Zoekt API so that Indexer can send callback messages to GitLab. It should be protected by JWT similar to /api/v4/internal/search/zoekt/:uuid/tasks

POST /api/v4/internal/search/zoekt/:uuid/callback

{
  "callback": "index",
  "success": true,
  "error": null,
  "payload": {
    "project_id": 10,
    "size_bytes": 1048576,
    "version": 1694429054
  }
}

I think in the first iteration we can log the requests. Later, we'll hook this up with other new Zoekt tables.

Edited by Dmitry Gruzd