Render charts in GitLab Issues using a Grafana URL
Problem to Solve
Our enterprise customers who are using GitLab monitoring (Prometheus) are making heavy use of Grafana visualizations. When there are incidents, we have found that incident responders are having to copy & paste screenshots of charts from Grafana in issues so that they can share knowledge and information with teammates as well as keep the issue up-to-date so that they can communicate the status of the incident to stakeholders.
We want to give them the ability to embed Grafana metrics in issues using a link. We want there to be two types of embeds:
- Static (i.e. set to a time frame) - this shows what was going on when the incident happened
- Live - updates as time progresses - this is helpful to communicate progress on the incident
This work contributes to the Incident Management Vision
- A Grafana admin generates a token for use in GitLab.
- A user with elevated permissions in GitLab (admin, owner) configures a group/project/instance for Grafana integration. Fields are:
- Grafana instance url.
- Grafana access token.
Embedding a chart
- An incident occurs.
- Incident issue is created (manually or automatically).
- User adds chart to issue by doing the following:
- In Grafana, user clicks 'Share' on the panel in question.
- They copy the link that appears in the Share dialog.
- They paste that link in the issue and hit Preview/Save.
- If the link is valid and the integration exists, a chart will render underneath the link.
- The chart is interactive, and displays with the same visual theme as other GitLab metrics charts. The link is clickable and takes the user to the chart in Grafana.
- Users can share panels that are backed by Prometheus data sources.
- The Prometheus data source is a simple pass-through in Grafana - so we can reuse our existing code for this.
- Other Grafana data sources have different formats. These will will require additional effort to support. There does appear to be a standard format that plugins may use for time-series data.
- We could show a specific error for non-Prometheus cases, or simply not expand the chart.
- Time series data only.
- Display as a line chart.
- Other chart types could be filtered out by the backend or frontend.
- Transfer basic metadata only.
- labels, titles, units.
- Panels include start + end time range.
- This is the default for the Grafana Share dialog.
Auth portion is done).Work required (assuming
- Implement Grafana Panel JSON interface. (Which data source am I interested in + Chart metadata).
- Implement Grafana Datasource interface. (How to make requests to the data source / what type is it).
- Implement Grafana Prometheus pass-through interface. (Fetch the data).
- Pull it all together into a new step in the GFM pipeline.
- Implement a new step in the frontend portion of the GFM pipeline.
- Implement a new Grafana embed component - similar to existing metrics embeds with some minor differences.
- Implement additional data processing.
- For Prometheus datasources: we can reuse existing frontend data processing code since we already support Prometheus charts.
- For non-Prometheus time-series data: low-to-medium complexity, some data manipulation will be required for use in our charts.
- Non-time-series data: Low if we already support the chart-type in question (e.g. Single-value). Higher effort if we need to implement a new chart.
Permissions and Security
What does success look like, and how can we measure that?
What is the type of buyer?
Links / references
MR Management & Status
|!17234 (merged) - BE Authentication||Pre-req to embeds.||Merged|
|!17330 - FE Authentication||Pre-req to embeds.||workflow::In review|
|!18210 (merged) - Adds FF||Ease of release.||Merged|
|!17675 (merged) - BE Refactor||Ease of development.||Merged|
|!18210 (merged) - BE for embeds (1)||Critical core feature work.||Merged|
|!18520 - BE for embeds (2)||Critical core feature work.||workflow::In review|
|!18285 - BE for embeds (3)||Critical core feature work.||workflow::In review|
|!18426 - FE for embeds||Core feature work. If shipped without merging, the user would need to refresh the page to see the chart and some legend labels would not be descriptive of the content in the chart.||workflow::In dev - WIP MR up|
|!18486 - FF removal||Final piece of dev work. Ease of release.||Blocked by all above. - WIP MR up|
|TBD - Docs MR||Critical. There are some caveats to this first iteration which will be necessary to document for usage of the feature.||workflow::In dev|
|!18536 - Feature test||Ease of maintenance. Non-blocking for release of feature.||workflow::In dev - WIP MR up|