Skip to content

Add concurrency in_progress log field and fix queue_length log's semantic

Quang-Minh Nguyen requested to merge qmnguyen0711/add-in-progress-log-field into master

The concurrency limiter attaches some log fields to gRPC logs. Those fields include useful information for operations. There is one log field called limiter.concurrency_queue_length. It tells the number of requests in the queue. That log field consists of in-progress requests and in-queue requests. Although the number of in-queue requests can be calculated from the concurrency configuration and the queue length, it's troublesome to do so.

This semantic reflects the internal implementation of the limiter. The limiter uses two systems: queue tokens and concurrency tokens. A request must acquire the queue token before acquiring the concurrency token

This MR adds limit.concurrency_in_progress log field and fixes the semantic of limit.concurrency_queue_length. This fix aligns with the corresponding Prometheus metrics.

Please note that the in_process and queue_length logs reflect the corresponding data at the time the request is queued, or is in-process. Unlike the gauge metric, it does not update the data afterward.

Under the limit

Screenshot_2023-08-15_at_19.00.13

Max wait time exceeded

Screenshot_2023-08-15_at_18.56.46

The waiting queue is full

Screenshot_2023-08-15_at_19.08.05

The waiting queue is disabled

Screenshot_2023-08-15_at_19.09.16

Edited by Quang-Minh Nguyen

Merge request reports