Skip reload on grafana dashboard sidecar

What does this MR do and why?

Grafana has a sidecar called grafana-sc-dashboard with the scope of watching over new dashboards and be sure that will be succesfully loaded. This mechanism is performed on a defined period of time ( into values via updateIntervalSeconds, by default is 30), but in the same time to avoid waiting too much, there is another components ( reloader) responsible to make an api call towards grafana once a new dashboard is detected ( /api/admin/provisioning/dashboards/reload).

In our case both of them are enabled, but looking into logs we could see that the sidecar is not able to trigger the reload being unauthorized to run this action. We do not see any impact for the dashboards loaded in grafana, because even the reloader does not work as expected, first mechanism is taking place and having a short period of time ( 30s) is enought to update the dashboards.

logger=context userId=0 orgId=0 uname= t=2026-08-12T10:54:36.144826965Z level=info msg="Request Completed" method=POST path=/api/admin/provisioning/dashboards/reload status=401 remote_addr=127.0.0.1 time_ms=1282 duration=1.282422114s size=102 referer= handler=/api/admin/provisioning/dashboards/reload status_source=server errorReason=Unauthorized errorMessageID=auth.unauthorized error="cannot authenticate request"
logger=context userId=0 orgId=0 uname= t=2026-08-12T10:54:37.949185936Z level=info msg="Request Completed" method=POST path=/api/admin/provisioning/dashboards/reload status=401 remote_addr=127.0.0.1 time_ms=895 duration=895.094949ms size=102 referer= handler=/api/admin/provisioning/dashboards/reload status_source=server errorReason=Unauthorized errorMessageID=auth.unauthorized error="cannot authenticate request"
logger=context userId=0 orgId=0 uname= t=2026-08-12T10:54:39.246671576Z level=info msg="Request Completed" method=POST path=/api/admin/provisioning/dashboards/reload status=401 remote_addr=127.0.0.1 time_ms=999 duration=999.451986ms size=102 referer= handler=/api/admin/provisioning/dashboards/reload status_source=server errorReason=Unauthorized errorMessageID=auth.unauthorized error="cannot authenticate request"
logger=context userId=0 orgId=0 uname= t=2026-08-12T10:54:40.453849031Z level=info msg="Request Completed" method=POST path=/api/admin/provisioning/dashboards/reload status=401 remote_addr=127.0.0.1 time_ms=908 duration=908.953169ms size=102 referer= handler=/api/admin/provisioning/dashboards/reload status_source=server errorReason=Unauthorized errorMessageID=auth.unauthorized error="cannot authenticate request"

But there are some cases where grafana is flooded by many requests from reload and ended up being unresponsive as we can notice in some sso tests where grafana service is unavailable. see:

https://gitlab.com/sylva-projects/sylva-core/-/jobs/15811317999

https://gitlab.com/sylva-projects/sylva-core/-/jobs/15855516388

The aim of this MR is to disable reloader and rely only on periodically updated for newest dashboards and in the same time I've increased the interval in order avoid having requests too often.

Also to explain why the reloader is not able to preform the request I was trying locally to replicate the process using same user and password (which for reloader are populated via env variables) and got:

 strict-transport-security: max-age=31536000; includeSubDomains
<
{"extra":null,"message":"Unauthorized","messageId":"auth.unauthorized","statusCode":401,"traceID":""}
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Connection #0 to host grafana.sylva left intact

In the end I conclude that is the expected behaviour because in grafana.ini configuration we explicit block the basic auth and I think we should keep like this, rely on the periodically updates and avoid possible malitious attacks:

[analytics]
check_for_updates = true
[auth]
disable_login_form = false
[auth.anonymous]
enabled = false
org_role = Viewer
[auth.basic]
enabled = false <= here

Just to verify, I've changed the auth.basic to true and I was able to perform the request:

< HTTP/2 200
< date: Thu, 13 Aug 2026 13:38:52 GMT
< content-type: application/json
< content-length: 40
< cache-control: no-store
< x-content-type-options: nosniff
< x-xss-protection: 1; mode=block
< strict-transport-security: max-age=31536000; includeSubDomains
<
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Connection #0 to host grafana.sylva left intact
logger=provisioning.dashboard t=2026-08-13T13:38:59.681790957Z level=info msg="starting to provision dashboards"
logger=provisioning.dashboard t=2026-08-13T13:39:00.18660441Z level=info msg="finished to provision dashboards"

Closes #3442 (closed).

Test coverage

CI configuration

Below you can choose test deployment variants to run in this MR's CI.

Click to open to CI configuration

Legend:

Icon Meaning Available values
☁️ Infra Provider capd, capo, capm3
🚀 Bootstrap Provider kubeadm (alias kadm), rke2, okd, ck8s
🐧 Node OS ubuntu, suse, na, leapmicro
🛠️ Deployment Options Deployment option list and description
🎬 Pipeline Scenarios Available scenario list and description
🟢 Enabled units Any available units name, by default apply to management and workload cluster. Can be prefixed by mgmt: or wkld: to be applied only to a specific cluster type
🔴 Disabled units Any available units name, by default apply to management and workload cluster. Can be prefixed by mgmt: or wkld: to be applied only to a specific cluster type
🏗️ Target platform Can be used to select specific deployment environment Available platform list and description
Pipeline control autorun, manual or blocking. Can be used to override global config and start a deployment pipeline the required way
  • 🎬 preview ☁️ capd 🚀 kadm 🐧 ubuntu

  • 🎬 preview ☁️ capo 🚀 rke2 🐧 suse

  • 🎬 preview ☁️ capm3 🚀 rke2 🐧 ubuntu

  • ☁️ capd 🚀 kadm 🛠️ light-deploy 🐧 ubuntu

  • ☁️ capd 🚀 rke2 🛠️ light-deploy 🐧 suse

  • ☁️ capo 🚀 rke2 🐧 suse

  • ☁️ capo 🚀 rke2 🐧 leapmicro

  • ☁️ capo 🚀 kadm 🐧 ubuntu

  • ☁️ capo 🚀 kadm 🐧 ubuntu 🟢 neuvector,mgmt:harbor

  • ☁️ capo 🚀 rke2 🎬 rolling-update 🛠️ ha 🐧 ubuntu

  • ☁️ capo 🚀 kadm 🎬 wkld-k8s-upgrade 🐧 ubuntu

  • ☁️ capo 🚀 rke2 🎬 rolling-update-no-wkld 🛠️ ha 🐧 suse

  • ☁️ capo 🚀 rke2 🎬 sylva-upgrade 🛠️ ha 🐧 ubuntu

  • ☁️ capo 🚀 rke2 🎬 sylva-upgrade-from-1.6.x 🛠️ ha,misc 🐧 ubuntu

  • ☁️ capo 🚀 rke2 🛠️ ha,misc 🐧 ubuntu

  • ☁️ capo 🚀 rke2 🛠️ misc 🐧 ubuntu 🟢 mgmt:harbor 🔴 neuvector

  • ☁️ capo 🚀 rke2 🛠️ ha,misc,openbao 🐧 suse

  • ☁️ capo 🚀 rke2 🐧 suse 🎬 upgrade-from-prev-tag

  • ☁️ capm3 🚀 rke2 🐧 suse

  • ☁️ capm3 🚀 kadm 🐧 ubuntu

  • ☁️ capm3 🚀 ck8s 🐧 ubuntu

  • ☁️ capm3 🚀 kadm 🎬 rolling-update-no-wkld 🛠️ ha,misc 🐧 ubuntu

  • ☁️ capm3 🚀 rke2 🎬 wkld-k8s-upgrade 🛠️ ha 🐧 suse

  • ☁️ capm3 🚀 kadm 🎬 rolling-update 🛠️ ha 🐧 ubuntu

  • ☁️ capm3 🚀 rke2 🎬 upgrade-from-prev-release-branch 🛠️ ha 🐧 suse

  • ☁️ capm3 🚀 rke2 🛠️ misc,ha 🐧 suse

  • ☁️ capm3 🚀 rke2 🎬 sylva-upgrade 🛠️ ha,misc 🐧 suse

  • ☁️ capm3 🚀 kadm 🎬 rolling-update 🛠️ ha 🐧 suse

  • ☁️ capm3 🚀 ck8s 🎬 rolling-update 🛠️ ha 🐧 ubuntu

  • ☁️ capm3 🚀 rke2|okd 🎬 no-update 🐧 ubuntu|na

  • ☁️ capm3 🚀 rke2 🐧 suse 🎬 upgrade-from-release-1.5

  • ☁️ capm3 🚀 rke2 🐧 suse 🎬 upgrade-to-main

Global config for deployment pipelines

  • autorun pipelines
  • allow failure on pipelines
  • record sylvactl events

Notes:

  • Enabling autorun will make deployment pipelines to be run automatically without human interaction
  • Disabling allow failure will make deployment pipelines mandatory for pipeline success.
  • if both autorun and allow failure are disabled, deployment pipelines will need manual triggering but will be blocking the pipeline

Be aware: after configuration change, pipeline is not triggered automatically. Please run it manually (by clicking the run pipeline button in Pipelines tab) or push new code.

Edited by Bogdan Antohe

Merge request reports

Loading
Loading