Skip to content

Zoekt Implement callback API

Ravi Kumar requested to merge 424126-implement-callback-api into master

What does this MR do and why?

Added an endpoint POST /api/v4/internal/search/zoekt/:uuid/callback where uuid is the zoekt_shard uuid. It accepts the following data:

requires :name, type: String, desc: 'Callback name'
requires :success, type: Boolean, desc: 'Set to true if the operation is successful'
optional :error, type: String, desc: 'Detailed error message'
requires :payload, type: Hash, desc: 'Data payload for the request'

For the first iteration, we are just logging all the requests to this API. If success is true then log as info else log as error. If somehow shard could not be found(which usually is not possible) we are logging as info with shard_id as nil and status unprocessable_entity

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

Create the token

  • Run the rails console
bundle exec rails c
secret_token =  Gitlab::Shell.secret_token
issuer =  API::Helpers::GITLAB_SHELL_JWT_ISSUER
jwt_token = JSONWebToken::HMACToken.new(secret_token).tap do |token|
  token.issuer = issuer
end
jwt_token.encoded

Tail the zoekt.log

tail -f log/zoekt.log

Testing for unathorized rquests:

  • Run the following curl command. And expect you get 401 Unauthorized 
curl --location 'http://127.0.0.1:3000/api/v4/internal/search/zoekt/00000000-0000-0000-0000-000000000000/callback' \
--header 'Content-Type: application/json' \
--data '{
    "name": "index",
    "success": true,
    "payload": {"a": 5 }
}'

Please replace jwt_encode_token_from_above_step for the following tests with the token you get from the above step.

Testing for shard could not be found:

  • Run the following curl command. And expect you get 422 Unprocessable Entity. And check that you get log entry with info and node_id nil 
curl --location 'http://127.0.0.1:3000/api/v4/internal/search/zoekt/dummy/callback' \
--header 'Gitlab-Shell-Api-Request: jwt_encode_token_from_above_step' \
--header 'Content-Type: application/json' \
--data '{
    "name": "index",
    "success": true,
    "payload": {"a": 5 }
}'

Testing for Success true:

  • Run the following curl command. And expect you get 202 Accepted. And check that you get log entry with info, node_id and payload 
curl --location 'http://127.0.0.1:3000/api/v4/internal/search/zoekt/dummy/callback' \
--header 'Gitlab-Shell-Api-Request: jwt_encode_token_from_above_step' \
--header 'Content-Type: application/json' \
--data '{
    "name": "index",
    "success": true,
    "payload": {"a": 5 }
}'

Testing for Success false:

  • Run the following curl command. And expect you get 202 Accepted. And check that you get log entry with error, node_id, and payload 
curl --location 'http://127.0.0.1:3000/api/v4/internal/search/zoekt/dummy/callback' \
--header 'Gitlab-Shell-Api-Request: jwt_encode_token_from_above_step' \
--header 'Content-Type: application/json' \
--data '{
    "name": "index",
    "success": false,
    "payload": {"a": 5 }
}'

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #424126 (closed)

Edited by Ravi Kumar

Merge request reports