Platform: Create Protobuf-over-gRPC receiver

Problem to solve

The Platform needs to have a gRPC service running which accepts Protobuf requests and returns the requested data from Clickhouse. The actual transformation process will be completed in Platform: Add query API transformation layer (#22 - closed), but we should create the gRPC structure for this transformation to take place.

Technical considerations

As per the design document, we must consider:

  • We will need to eventually support streams for instances when a lot of data is being queried.
  • We require all queries to have a sorting field (this can be defaulted in the proto definition or query transformation).
  • We require all aggregation queries to have a grouping field (this can be defaulted in the proto definition or query transformation).
  • We must use keyset pagination.
  • We must version the gRPC service (see Microsoft's example guide).
  • We will be adding shared ENV token validation to all gRPC requests.
  • We will be recording gRPC request metrics.
  • .proto files must be saved to the platform repo and rails monolith.

The following response codes must be returned by gRPC:

  • 200 OK - On success response
  • 400 INVALID_ARGUMENT - When a query is malformed
  • 401 UNAUTHENTICATED - When a user is not authenticated
  • 403 PERMISSION_DENIED - When a user is authenticated but doesn't have permission to query a particular resource
  • 404 NOT_FOUND - When a request is sent to an unknown endpoint
  • 429 RESOURCE_EXHAUSTED - When a request is trying to retrieve more data than the API can handle (querying all the data we have on every issue since time immemorial, for instance), or when rate limits are exceeded
  • 500 INTERNAL - When the server fails unexpectedly
Edited by Robert Hunt