Multiple custom variables defined in yml file breaks metrics dashboard
Summary
As for %13.0, the Metrics dashboard supports adding templating variables in the yml file (docs here). These variables are rendered on the UI as text inputs and dropdown elements and the values are eventually passed on to the Prometheus proxy API.
However, it was found out that if multiple variables are defined in a dashboard yml file, it results in charts not showing data. The reason is that the variables were not substituted as expected.
Repro
- Create a metric dashboard with at least 2 templating variables.
- Use these templating variables in the metrics query defined in the same yml file.
- After the pipeline runs, navigate to the metrics dashboard.
- Notice that the charts will be rendered empty.
Solution
In the current approach, the backend depends on the order in which the variables are passed as URL params. After extensive research, we found out that the URL params can be re-ordered by some infrastructure-related middlewares resulting in queries with unsubstituted variables.
The fix is to make it the backend logic order-agnostic.
Technical details
Current apporach
variables[]=key1
variables[]=value1
variables[]=key2
variables[]=value2
Proposed apporach
variables[key1]=value1
variables[key2]=value2