Skip to content

NATS: Add readiness details for GA maturity

We're positioning NATS as a solution for queuing/buffering data within our data ingest pipelines, starting with those for Data Insights Platform.

Overview_v2_no_OTEL

This effort collectively leads to building a more generic abstraction around ingesting, processing & querying data for analytical features at GitLab — which we have started calling Data Insights Platform. We propose NATS to form the data backbone of this system/platform.

Most of this development comes as a result of our work within the Product Data Usage Working Group; these bits focused at consolidating the various ways we deal with analytical data within the product.

Additional context: We have had multiple discussions about what technology to use here, for example here. NATS stands out given its minimal footprint, ease of distribution and its ability to both be embeddable within the Product and scale out as a distributed system as needed.

More updates around the background work in this issue: #125

First iteration of Data Insights Platform will be used for sending product usage information. See epic.

Edited by Arun Sori

Merge request reports

Loading