Grafana crashing when using external InfluxDB data source

Summary

Grafana provided with GitLab Omnibus crashes when attempting to connect to external InfluxDB data source after GitLab upgrade to version 15.6

Steps to reproduce

  1. Navigate to gitlab.example.com/-/grafana, log in, go to data sources configuration,
  2. Add new InfluxDB data source,
  3. Specify InfluxDB data source configuration:
    • Query language: Flux
    • URL: https://influxdb.example.com:8086
    • Basic auth,
    • Correct organization,
    • Valid token,
    • Valid default bucket
  4. Click Save & Test.

What is the current bug behavior?

After clicking Save & Test UI displays two messages: Data source updated and Error reading InfluxDB. Grafana logs report SIGSEGV and the service is restarted.

What is the expected correct behavior?

After clicking Save & Test UI displays two messages: Data source updated and x buckets found. Logs are clean and newly created data source can be used in dashboards.

Relevant logs and/or screenshots

Grafana log immediately after clicking Save & Test: 
2022-12-06_11:05:48.53124 unexpected fault address 0x0
2022-12-06_11:05:48.53145 fatal error: fault
2022-12-06_11:05:48.53342 [signal SIGSEGV: segmentation violation code=0x80 addr=0x0 pc=0x46f09f]
2022-12-06_11:05:48.53349 
2022-12-06_11:05:48.53352 goroutine 238 [running]:
2022-12-06_11:05:48.53356 runtime.throw({0x2293bd3?, 0x18?})
2022-12-06_11:05:48.53362 	/usr/local/go/src/runtime/panic.go:992 +0x71 fp=0xc0008306f0 sp=0xc0008306c0 pc=0x43eb11
2022-12-06_11:05:48.53367 runtime.sigpanic()
2022-12-06_11:05:48.53370 	/usr/local/go/src/runtime/signal_unix.go:825 +0x2ec fp=0xc000830740 sp=0xc0008306f0 pc=0x454d0c
2022-12-06_11:05:48.53374 aeshashbody()
2022-12-06_11:05:48.53378 	/usr/local/go/src/runtime/asm_amd64.s:1343 +0x39f fp=0xc000830748 sp=0xc000830740 pc=0x46f09f
2022-12-06_11:05:48.53384 runtime.mapiternext(0xc0009c4500)
2022-12-06_11:05:48.53388 	/usr/local/go/src/runtime/map.go:934 +0x2cb fp=0xc0008307b8 sp=0xc000830748 pc=0x41890b
2022-12-06_11:05:48.53392 runtime.mapiterinit(0x0?, 0x8?, 0x1?)
2022-12-06_11:05:48.53397 	/usr/local/go/src/runtime/map.go:861 +0x228 fp=0xc0008307d8 sp=0xc0008307b8 pc=0x4185e8
2022-12-06_11:05:48.53400 reflect.mapiterinit(0x2?, 0xc000830860?, 0x467fc5?)
2022-12-06_11:05:48.53404 	/usr/local/go/src/runtime/map.go:1373 +0x19 fp=0xc000830800 sp=0xc0008307d8 pc=0x46b6f9
2022-12-06_11:05:48.53407 github.com/modern-go/reflect2.(*UnsafeMapType).UnsafeIterate(...)
2022-12-06_11:05:48.53411 	/root/go/pkg/mod/github.com/modern-go/reflect2@v1.0.1/unsafe_map.go:112
2022-12-06_11:05:48.53414 github.com/json-iterator/go.(*sortKeysMapEncoder).Encode(0xc000ce4420, 0xc0000be780, 0xc000dd32c0)
2022-12-06_11:05:48.53418 	/root/go/pkg/mod/github.com/json-iterator/go@v1.1.10/reflect_map.go:291 +0x225 fp=0xc000830970 sp=0xc000830800 pc=0xf30385
2022-12-06_11:05:48.53422 github.com/json-iterator/go.(*structFieldEncoder).Encode(0xc000ce44e0, 0x1c60f50?, 0xc000dd32c0)
2022-12-06_11:05:48.53426 	/root/go/pkg/mod/github.com/json-iterator/go@v1.1.10/reflect_struct_encoder.go:110 +0x56 fp=0xc0008309e8 sp=0xc000830970 pc=0xf3dad6
2022-12-06_11:05:48.53430 github.com/json-iterator/go.(*structEncoder).Encode(0xc000ce4570, 0x0?, 0xc000dd32c0)
2022-12-06_11:05:48.53433 	/root/go/pkg/mod/github.com/json-iterator/go@v1.1.10/reflect_struct_encoder.go:158 +0x765 fp=0xc000830ad0 sp=0xc0008309e8 pc=0xf3e4e5
2022-12-06_11:05:48.53438 github.com/json-iterator/go.(*OptionalEncoder).Encode(0xc0004aa1e0?, 0x0?, 0x0?)
2022-12-06_11:05:48.53442 	/root/go/pkg/mod/github.com/json-iterator/go@v1.1.10/reflect_optional.go:70 +0xa4 fp=0xc000830b20 sp=0xc000830ad0 pc=0xf356e4
2022-12-06_11:05:48.53445 github.com/json-iterator/go.(*onePtrEncoder).Encode(0xc000761460, 0xc0000be780, 0xc000ecbfb0?)
2022-12-06_11:05:48.53449 	/root/go/pkg/mod/github.com/json-iterator/go@v1.1.10/reflect.go:219 +0x82 fp=0xc000830b58 sp=0xc000830b20 pc=0xf28842
2022-12-06_11:05:48.53452 github.com/json-iterator/go.(*Stream).WriteVal(0xc000dd32c0, {0x1d1a3a0, 0xc0000be780})
2022-12-06_11:05:48.53456 	/root/go/pkg/mod/github.com/json-iterator/go@v1.1.10/reflect.go:98 +0x158 fp=0xc000830bc8 sp=0xc000830b58 pc=0xf27b58
2022-12-06_11:05:48.53460 github.com/json-iterator/go.(*Encoder).Encode(0xc000a08348, {0x1d1a3a0?, 0xc0000be780?})
2022-12-06_11:05:48.53463 	/root/go/pkg/mod/github.com/json-iterator/go@v1.1.10/adapter.go:127 +0x32 fp=0xc000830c18 sp=0xc000830bc8 pc=0xf13912
2022-12-06_11:05:48.53467 github.com/grafana/grafana/pkg/api/response.StreamingResponse.WriteTo({{0x1d1a3a0?, 0xc0000be780?}, 0x1f84340?, 0xc000ecbf20?}, 0xc0003da4c0)
2022-12-06_11:05:48.53470 	/var/cache/omnibus/src/grafana/pkg/api/response/response.go:111 +0x1d7 fp=0xc000830d20 sp=0xc000830c18 pc=0x104ed17
2022-12-06_11:05:48.53475 github.com/grafana/grafana/pkg/api/response.(*StreamingResponse).WriteTo(0x1f84340?, 0xc0000a0c90?)
2022-12-06_11:05:48.53479 	:1 +0x45 fp=0xc000830d58 sp=0xc000830d20 pc=0x104f625
2022-12-06_11:05:48.53484 github.com/grafana/grafana/pkg/api/routing.Wrap.func1(0xc0003da4c0)
2022-12-06_11:05:48.53488 	/var/cache/omnibus/src/grafana/pkg/api/routing/routing.go:25 +0x9d fp=0xc000830d90 sp=0xc000830d58 pc=0x12dd1fd
2022-12-06_11:05:48.53492 runtime.call16(0xc00002da10, 0xc0009f2318, 0x0, 0x0, 0x0, 0x8, 0xc0008312c0)
2022-12-06_11:05:48.53495 	/usr/local/go/src/runtime/asm_amd64.s:701 +0x49 fp=0xc000830db0 sp=0xc000830d90 pc=0x46ffc9
2022-12-06_11:05:48.53499 runtime.reflectcall(0x2270820?, 0xc0003da4c0?, 0x1?, 0x22b48bc?, 0x0?, 0x12?, 0x2270820?)
2022-12-06_11:05:48.53502 	:1 +0x3c fp=0xc000830df0 sp=0xc000830db0 pc=0x47453c
2022-12-06_11:05:48.53506 reflect.Value.call({0x1d7f740?, 0xc0009f2318?, 0x7fceaf01da68?}, {0x2291c88, 0x4}, {0xc00000f7b8, 0x1, 0x0?})
2022-12-06_11:05:48.53509 	/usr/local/go/src/reflect/value.go:556 +0x845 fp=0xc0008313e0 sp=0xc000830df0 pc=0x49b3c5
2022-12-06_11:05:48.53513 reflect.Value.Call({0x1d7f740?, 0xc0009f2318?, 0x2270820?}, {0xc00000f7b8, 0x1, 0x1})
2022-12-06_11:05:48.53518 	/usr/local/go/src/reflect/value.go:339 +0xbf fp=0xc000831458 sp=0xc0008313e0 pc=0x49a93f
2022-12-06_11:05:48.53553 github.com/go-macaron/inject.(*injector).callInvoke(0x222c7d22223a2264?, {0x1d7f740?, 0xc0009f2318?}, {0x26823f8?, 0x1d7f740?}, 0x1?)
2022-12-06_11:05:48.53557 	/root/go/pkg/mod/github.com/go-macaron/inject@v0.0.0-20160627170012-d8a0b8677191/inject.go:177 +0x127 fp=0xc0008314e8 sp=0xc000831458 pc=0xe158c7
2022-12-06_11:05:48.53561 github.com/go-macaron/inject.(*injector).Invoke(0x6574617473222c31?, {0x1d7f740?, 0xc0009f2318})
2022-12-06_11:05:48.53564 	/root/go/pkg/mod/github.com/go-macaron/inject@v0.0.0-20160627170012-d8a0b8677191/inject.go:137 +0xb5 fp=0xc000831568 sp=0xc0008314e8 pc=0xe15515
2022-12-06_11:05:48.53568 gopkg.in/macaron%2ev1.(*Context).run(0xc0002566c0)
2022-12-06_11:05:48.53572 	/var/cache/omnibus/src/grafana/pkg/macaron/context.go:125 +0x73 fp=0xc0008315c0 sp=0xc000831568 pc=0xe17533
2022-12-06_11:05:48.53576 gopkg.in/macaron%2ev1.(*Context).Next(...)
2022-12-06_11:05:48.53580 	/var/cache/omnibus/src/grafana/pkg/macaron/context.go:115
2022-12-06_11:05:48.53584 github.com/grafana/grafana/pkg/middleware.RequestMetrics.func1.1({0x7fce84ece958?, 0xc0000a6d20}, 0xc0009d6600, 0xc0002566c0)
2022-12-06_11:05:48.53589 	/var/cache/omnibus/src/grafana/pkg/middleware/request_metrics.go:54 +0x134 fp=0xc000831ca0 sp=0xc0008315c0 pc=0x1330054
2022-12-06_11:05:48.53593 runtime.call32(0xc000bff770, 0xc0009eeb40, 0x0, 0x0, 0x0, 0x20, 0xc0008321e0)
2022-12-06_11:05:48.53596 	/usr/local/go/src/runtime/asm_amd64.s:702 +0x49 fp=0xc000831cd0 sp=0xc000831ca0 pc=0x470069
2022-12-06_11:05:48.53600 runtime.reflectcall(0x226f5e0?, 0xc0002566c0?, 0x4?, 0x22b48bc?, 0x0?, 0x12?, 0x226f5e0?)
2022-12-06_11:05:48.53604 	:1 +0x3c fp=0xc000831d10 sp=0xc000831cd0 pc=0x47453c
2022-12-06_11:05:48.53608 reflect.Value.call({0x1e7fde0?, 0xc0009eeb40?, 0x7fceaf01da68?}, {0x2291c88, 0x4}, {0xc0000a7040, 0x3, 0x2?})
2022-12-06_11:05:48.53612 	/usr/local/go/src/reflect/value.go:556 +0x845 fp=0xc000832300 sp=0xc000831d10 pc=0x49b3c5
2022-12-06_11:05:48.53615 reflect.Value.Call({0x1e7fde0?, 0xc0009eeb40?, 0x226f5e0?}, {0xc0000a7040, 0x3, 0x3})
2022-12-06_11:05:48.53619 	/usr/local/go/src/reflect/value.go:339 +0xbf fp=0xc000832378 sp=0xc000832300 pc=0x49a93f
2022-12-06_11:05:48.53623 github.com/go-macaron/inject.(*injector).callInvoke(0xc000060c00?, {0x1e7fde0?, 0xc0009eeb40?}, {0x26823f8?, 0x1e7fde0?}, 0x3?)
2022-12-06_11:05:48.53627 	/root/go/pkg/mod/github.com/go-macaron/inject@v0.0.0-20160627170012-d8a0b8677191/inject.go:177 +0x127 fp=0xc000832408 sp=0xc000832378 pc=0xe158c7
2022-12-06_11:05:48.53632 github.com/go-macaron/inject.(*injector).Invoke(0xc000060c00?, {0x1e7fde0?, 0xc0009eeb40})
2022-12-06_11:05:48.53636 	/root/go/pkg/mod/github.com/go-macaron/inject@v0.0.0-20160627170012-d8a0b8677191/inject.go:137 +0xb5 fp=0xc000832488 sp=0xc000832408 pc=0xe15515
2022-12-06_11:05:48.53640 gopkg.in/macaron%2ev1.(*Context).run(0xc0002566c0)
2022-12-06_11:05:48.53643 	/var/cache/omnibus/src/grafana/pkg/macaron/context.go:125 +0x73 fp=0xc0008324e0 sp=0xc000832488 pc=0xe17533
2022-12-06_11:05:48.53650 gopkg.in/macaron%2ev1.(*Context).Next(...)
2022-12-06_11:05:48.53654 	/var/cache/omnibus/src/grafana/pkg/macaron/context.go:115
2022-12-06_11:05:48.53657 github.com/grafana/grafana/pkg/middleware.RequestTracing.func1({0x7fce84ece958?, 0xc0000a6d20}, 0xc0009d6600, 0xc0002566c0)
2022-12-06_11:05:48.53661 	/var/cache/omnibus/src/grafana/pkg/middleware/request_tracing.go:25 +0x4c5 fp=0xc000832628 sp=0xc0008324e0 pc=0x1331a45
2022-12-06_11:05:48.53665 runtime.call32(0xc000bff770, 0xc0009f2330, 0x0, 0x0, 0x0, 0x20, 0xc000832b68)
2022-12-06_11:05:48.53669 	/usr/local/go/src/runtime/asm_amd64.s:702 +0x49 fp=0xc000832658 sp=0xc000832628 pc=0x470069
2022-12-06_11:05:48.53673 runtime.reflectcall(0x226f5e0?, 0xc0002566c0?, 0x4?, 0x22b48bc?, 0x0?, 0x12?, 0x226f5e0?)
2022-12-06_11:05:48.53678 	:1 +0x3c fp=0xc000832698 sp=0xc000832658 pc=0x47453c
2022-12-06_11:05:48.53682 reflect.Value.call({0x1e7fde0?, 0xc0009f2330?, 0x7fceaf01da68?}, {0x2291c88, 0x4}, {0xc0000a6ff0, 0x3, 0xc000d487a0?})
2022-12-06_11:05:48.53686 	/usr/local/go/src/reflect/value.go:556 +0x845 fp=0xc000832c88 sp=0xc000832698 pc=0x49b3c5
2022-12-06_11:05:48.53689 reflect.Value.Call({0x1e7fde0?, 0xc0009f2330?, 0x226f5e0?}, {0xc0000a6ff0, 0x3, 0x3})
2022-12-06_11:05:48.53693 	/usr/local/go/src/reflect/value.go:339 +0xbf fp=0xc000832d00 sp=0xc000832c88 pc=0x49a93f
2022-12-06_11:05:48.53697 github.com/go-macaron/inject.(*injector).callInvoke(0xc000ef2db8?, {0x1e7fde0?, 0xc0009f2330?}, {0x26823f8?, 0x1e7fde0?}, 0x3?)
2022-12-06_11:05:48.53701 	/root/go/pkg/mod/github.com/go-macaron/inject@v0.0.0-20160627170012-d8a0b8677191/inject.go:177 +0x127 fp=0xc000832d90 sp=0xc000832d00 pc=0xe158c7
2022-12-06_11:05:48.53705 github.com/go-macaron/inject.(*injector).Invoke(0xc000d6a6c0?, {0x1e7fde0?, 0xc0009f2330})
2022-12-06_11:05:48.53709 	/root/go/pkg/mod/github.com/go-macaron/inject@v0.0.0-20160627170012-d8a0b8677191/inject.go:137 +0xb5 fp=0xc000832e10 sp=0xc000832d90 pc=0xe15515
2022-12-06_11:05:48.53714 gopkg.in/macaron%2ev1.(*Context).run(0xc0002566c0)
2022-12-06_11:05:48.53717 	/var/cache/omnibus/src/grafana/pkg/macaron/context.go:125 +0x73 fp=0xc000832e68 sp=0xc000832e10 pc=0xe17533
2022-12-06_11:05:48.53721 gopkg.in/macaron%2ev1.(*Context).Next(...)
2022-12-06_11:05:48.53725 	/var/cache/omnibus/src/grafana/pkg/macaron/context.go:115
2022-12-06_11:05:48.53729 github.com/grafana/grafana/pkg/middleware.Recovery.func1(0x10?)
2022-12-06_11:05:48.53733 	/var/cache/omnibus/src/grafana/pkg/middleware/recovery.go:166 +0x6b fp=0xc000832ea8 sp=0xc000832e68 pc=0x132f1eb
2022-12-06_11:05:48.53737 gopkg.in/macaron%2ev1.ContextInvoker.Invoke(0x226f5e0?, {0xc000d48150?, 0x226f5e0?, 0x203000?})
2022-12-06_11:05:48.53740 	/var/cache/omnibus/src/grafana/pkg/macaron/context.go:81 +0x3c fp=0xc000832ed0 sp=0xc000832ea8 pc=0xe173dc
2022-12-06_11:05:48.53744 github.com/go-macaron/inject.(*injector).fastInvoke(0x120?, {0x7fce84ece988?, 0xc000210020?}, {0x26823f8?, 0x1efd560?}, 0x1?)
2022-12-06_11:05:48.53748 	/root/go/pkg/mod/github.com/go-macaron/inject@v0.0.0-20160627170012-d8a0b8677191/inject.go:157 +0x95 fp=0xc000832f40 sp=0xc000832ed0 pc=0xe15635
2022-12-06_11:05:48.53752 github.com/go-macaron/inject.(*injector).Invoke(0x1?, {0x1efd560?, 0xc000210020})
2022-12-06_11:05:48.53756 	/root/go/pkg/mod/github.com/go-macaron/inject@v0.0.0-20160627170012-d8a0b8677191/inject.go:135 +0xfd fp=0xc000832fc0 sp=0xc000832f40 pc=0xe1555d
2022-12-06_11:05:48.53761 gopkg.in/macaron%2ev1.(*Context).run(0xc0002566c0)
2022-12-06_11:05:48.53765 	/var/cache/omnibus/src/grafana/pkg/macaron/context.go:125 +0x73 fp=0xc000833018 sp=0xc000832fc0 pc=0xe17533
2022-12-06_11:05:48.53769 gopkg.in/macaron%2ev1.(*Context).Next(...)
2022-12-06_11:05:48.53773 	/var/cache/omnibus/src/grafana/pkg/macaron/context.go:115
2022-12-06_11:05:48.53777 github.com/grafana/grafana/pkg/middleware.Logger.func1({0x7fce84ece958?, 0xc0000a6d20}, 0xc0009d6600, 0xc0002566c0)
2022-12-06_11:05:48.53782 	/var/cache/omnibus/src/grafana/pkg/middleware/logger.go:35 +0x12f fp=0xc000833108 sp=0xc000833018 pc=0x132d08f
2022-12-06_11:05:48.53789 runtime.call32(0xc000bff770, 0xc000210000, 0x0, 0x0, 0x0, 0x20, 0xc000833648)
2022-12-06_11:05:48.53793 	/usr/local/go/src/runtime/asm_amd64.s:702 +0x49 fp=0xc000833138 sp=0xc000833108 pc=0x470069
2022-12-06_11:05:48.53797 runtime.reflectcall(0x226f5e0?, 0xc0002566c0?, 0x4?, 0x22b48bc?, 0x0?, 0x12?, 0x226f5e0?)
2022-12-06_11:05:48.53801 	:1 +0x3c fp=0xc000833178 sp=0xc000833138 pc=0x47453c
2022-12-06_11:05:48.53805 reflect.Value.call({0x1e7fde0?, 0xc000210000?, 0x13?}, {0x2291c88, 0x4}, {0xc0000a6d70, 0x3, 0x3?})
2022-12-06_11:05:48.53810 	/usr/local/go/src/reflect/value.go:556 +0x845 fp=0xc000833768 sp=0xc000833178 pc=0x49b3c5
2022-12-06_11:05:48.53814 reflect.Value.Call({0x1e7fde0?, 0xc000210000?, 0x226f5e0?}, {0xc0000a6d70, 0x3, 0x3})
2022-12-06_11:05:48.53818 	/usr/local/go/src/reflect/value.go:339 +0xbf fp=0xc0008337e0 sp=0xc000833768 pc=0x49a93f
2022-12-06_11:05:48.53821 github.com/go-macaron/inject.(*injector).callInvoke(0x16?, {0x1e7fde0?, 0xc000210000?}, {0x26823f8?, 0x1e7fde0?}, 0x3?)
2022-12-06_11:05:48.53825 	/root/go/pkg/mod/github.com/go-macaron/inject@v0.0.0-20160627170012-d8a0b8677191/inject.go:177 +0x127 fp=0xc000833870 sp=0xc0008337e0 pc=0xe158c7
2022-12-06_11:05:48.53829 github.com/go-macaron/inject.(*injector).Invoke(0xc000060c00?, {0x1e7fde0?, 0xc000210000})
2022-12-06_11:05:48.53833 	/root/go/pkg/mod/github.com/go-macaron/inject@v0.0.0-20160627170012-d8a0b8677191/inject.go:137 +0xb5 fp=0xc0008338f0 sp=0xc000833870 pc=0xe15515
2022-12-06_11:05:48.53837 gopkg.in/macaron%2ev1.(*Context).run(0xc0002566c0)
2022-12-06_11:05:48.53841 	/var/cache/omnibus/src/grafana/pkg/macaron/context.go:125 +0x73 fp=0xc000833948 sp=0xc0008338f0 pc=0xe17533
2022-12-06_11:05:48.53845 gopkg.in/macaron%2ev1.(*Router).Handle.func1({0x266f7e8?, 0xc0002ac700?}, 0x2290cd5?, 0x0)
2022-12-06_11:05:48.53850 	/var/cache/omnibus/src/grafana/pkg/macaron/router.go:187 +0x285 fp=0xc0008339d8 sp=0xc000833948 pc=0xe22305
2022-12-06_11:05:48.53854 gopkg.in/macaron%2ev1.(*Router).ServeHTTP(0xc00095e7e0, {0x266f7e8, 0xc0002ac700}, 0xc0009d6600)
2022-12-06_11:05:48.53858 	/var/cache/omnibus/src/grafana/pkg/macaron/router.go:295 +0x1b6 fp=0xc000833a38 sp=0xc0008339d8 pc=0xe234b6
2022-12-06_11:05:48.53862 gopkg.in/macaron%2ev1.(*Macaron).ServeHTTP(0xc0008df080, {0x266f7e8, 0xc0002ac700}, 0xc0009d6600)
2022-12-06_11:05:48.53866 	/var/cache/omnibus/src/grafana/pkg/macaron/macaron.go:220 +0x165 fp=0xc000833a98 sp=0xc000833a38 pc=0xe1bfe5
2022-12-06_11:05:48.53870 net/http.serverHandler.ServeHTTP({0x2669138?}, {0x266f7e8, 0xc0002ac700}, 0xc0009d6600)
2022-12-06_11:05:48.53873 	/usr/local/go/src/net/http/server.go:2916 +0x43b fp=0xc000833b58 sp=0xc000833a98 pc=0x6f6efb
2022-12-06_11:05:48.53877 net/http.(*conn).serve(0xc0004aae60, {0x2670ba8, 0xc00053fda0})
2022-12-06_11:05:48.53881 	/usr/local/go/src/net/http/server.go:1966 +0x5d7 fp=0xc000833fb8 sp=0xc000833b58 pc=0x6f23b7
2022-12-06_11:05:48.53885 net/http.(*Server).Serve.func3()
2022-12-06_11:05:48.53889 	/usr/local/go/src/net/http/server.go:3071 +0x2e fp=0xc000833fe0 sp=0xc000833fb8 pc=0x6f784e
2022-12-06_11:05:48.53893 runtime.goexit()
2022-12-06_11:05:48.53898 	/usr/local/go/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc000833fe8 sp=0xc000833fe0 pc=0x471ca1
2022-12-06_11:05:48.53902 created by net/http.(*Server).Serve
2022-12-06_11:05:48.53906 	/usr/local/go/src/net/http/server.go:3071 +0x4db
2022-12-06_11:05:48.53909 
2022-12-06_11:05:48.53913 goroutine 1 [semacquire, 20 minutes]:
2022-12-06_11:05:48.53917 sync.runtime_Semacquire(0x22c0073?)
2022-12-06_11:05:48.53922 	/usr/local/go/src/runtime/sema.go:56 +0x25
2022-12-06_11:05:48.53926 sync.(*WaitGroup).Wait(0x0?)
2022-12-06_11:05:48.53930 	/usr/local/go/src/sync/waitgroup.go:136 +0x52
2022-12-06_11:05:48.53934 golang.org/x/sync/errgroup.(*Group).Wait(0xc000339980)
2022-12-06_11:05:48.53938 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:40 +0x27
2022-12-06_11:05:48.53942 github.com/grafana/grafana/pkg/server.(*Server).Run.func2()
2022-12-06_11:05:48.53947 	/var/cache/omnibus/src/grafana/pkg/server/server.go:198 +0x5a
2022-12-06_11:05:48.53951 github.com/grafana/grafana/pkg/server.(*Server).Run(0xc000292750)
2022-12-06_11:05:48.53955 	/var/cache/omnibus/src/grafana/pkg/server/server.go:208 +0x21d
2022-12-06_11:05:48.53959 main.executeServer({0x7ffff44e3eca, 0x23}, {0x0, 0x0}, {0x0, 0x0}, {0x2298ba5, 0x7}, 0x226e540?)
2022-12-06_11:05:48.53963 	/var/cache/omnibus/src/grafana/pkg/cmd/grafana-server/main.go:161 +0x6e9
2022-12-06_11:05:48.53967 main.main()
2022-12-06_11:05:48.53971 	/var/cache/omnibus/src/grafana/pkg/cmd/grafana-server/main.go:98 +0x5c5
2022-12-06_11:05:48.53976 
2022-12-06_11:05:48.53980 goroutine 122 [select]:
2022-12-06_11:05:48.53983 go.opencensus.io/stats/view.(*worker).start(0xc0003ca300)
2022-12-06_11:05:48.53987 	/root/go/pkg/mod/go.opencensus.io@v0.22.6/stats/view/worker.go:276 +0xad
2022-12-06_11:05:48.53991 created by go.opencensus.io/stats/view.init.0
2022-12-06_11:05:48.53995 	/root/go/pkg/mod/go.opencensus.io@v0.22.6/stats/view/worker.go:34 +0x8d
2022-12-06_11:05:48.53999 
2022-12-06_11:05:48.54002 goroutine 125 [select, 2 minutes]:
2022-12-06_11:05:48.54006 github.com/patrickmn/go-cache.(*janitor).Run(0xc0008c8130, 0xc00003f2c0?)
2022-12-06_11:05:48.54010 	/root/go/pkg/mod/github.com/patrickmn/go-cache@v2.1.0+incompatible/cache.go:1079 +0x85
2022-12-06_11:05:48.54014 created by github.com/patrickmn/go-cache.runJanitor
2022-12-06_11:05:48.54018 	/root/go/pkg/mod/github.com/patrickmn/go-cache@v2.1.0+incompatible/cache.go:1099 +0xed
2022-12-06_11:05:48.54022 
2022-12-06_11:05:48.54026 goroutine 126 [select, 20 minutes]:
2022-12-06_11:05:48.54029 database/sql.(*DB).connectionOpener(0xc0008d0820, {0x2670b00, 0xc0008be1c0})
2022-12-06_11:05:48.54033 	/usr/local/go/src/database/sql/sql.go:1226 +0x8d
2022-12-06_11:05:48.54037 created by database/sql.OpenDB
2022-12-06_11:05:48.54041 	/usr/local/go/src/database/sql/sql.go:794 +0x18d
2022-12-06_11:05:48.54045 
2022-12-06_11:05:48.54049 goroutine 127 [select, 20 minutes]:
2022-12-06_11:05:48.54053 database/sql.(*DB).connectionCleaner(0xc0008d0820, 0x0?)
2022-12-06_11:05:48.54056 	/usr/local/go/src/database/sql/sql.go:1069 +0xa7
2022-12-06_11:05:48.54061 created by database/sql.(*DB).startCleanerLocked
2022-12-06_11:05:48.54065 	/usr/local/go/src/database/sql/sql.go:1056 +0xff
2022-12-06_11:05:48.54068 
2022-12-06_11:05:48.54072 goroutine 146 [select, 20 minutes]:
2022-12-06_11:05:48.54076 github.com/patrickmn/go-cache.(*janitor).Run(0xc0009e3a20, 0x0?)
2022-12-06_11:05:48.54079 	/root/go/pkg/mod/github.com/patrickmn/go-cache@v2.1.0+incompatible/cache.go:1079 +0x85
2022-12-06_11:05:48.54083 created by github.com/patrickmn/go-cache.runJanitor
2022-12-06_11:05:48.54087 	/root/go/pkg/mod/github.com/patrickmn/go-cache@v2.1.0+incompatible/cache.go:1099 +0xed
2022-12-06_11:05:48.54091 
2022-12-06_11:05:48.54095 goroutine 147 [select]:
2022-12-06_11:05:48.54099 github.com/grafana/grafana/pkg/services/alerting.(*Ticker).run(0xc000bb6410)
2022-12-06_11:05:48.54102 	/var/cache/omnibus/src/grafana/pkg/services/alerting/ticker.go:55 +0x1a7
2022-12-06_11:05:48.54106 created by github.com/grafana/grafana/pkg/services/alerting.NewTicker
2022-12-06_11:05:48.54110 	/var/cache/omnibus/src/grafana/pkg/services/alerting/ticker.go:39 +0x185
2022-12-06_11:05:48.54114 
2022-12-06_11:05:48.54118 goroutine 136 [select, 20 minutes]:
2022-12-06_11:05:48.54122 github.com/grafana/grafana/pkg/services/alerting.(*AlertEngine).runJobDispatcher(0xc0002565a0, {0x2670b00, 0xc0000a3040})
2022-12-06_11:05:48.54126 	/var/cache/omnibus/src/grafana/pkg/services/alerting/engine.go:99 +0x105
2022-12-06_11:05:48.54130 github.com/grafana/grafana/pkg/services/alerting.(*AlertEngine).Run.func2()
2022-12-06_11:05:48.54142 	/var/cache/omnibus/src/grafana/pkg/services/alerting/engine.go:64 +0x25
2022-12-06_11:05:48.54147 golang.org/x/sync/errgroup.(*Group).Go.func1()
2022-12-06_11:05:48.54151 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67
2022-12-06_11:05:48.54157 created by golang.org/x/sync/errgroup.(*Group).Go
2022-12-06_11:05:48.54161 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d
2022-12-06_11:05:48.54165 
2022-12-06_11:05:48.54169 goroutine 135 [select]:
2022-12-06_11:05:48.54173 github.com/grafana/grafana/pkg/services/alerting.(*AlertEngine).alertingTicker(0xc0002565a0, {0x2670b00, 0xc0000a3040})
2022-12-06_11:05:48.54177 	/var/cache/omnibus/src/grafana/pkg/services/alerting/engine.go:80 +0x145
2022-12-06_11:05:48.54182 github.com/grafana/grafana/pkg/services/alerting.(*AlertEngine).Run.func1()
2022-12-06_11:05:48.54186 	/var/cache/omnibus/src/grafana/pkg/services/alerting/engine.go:63 +0x25
2022-12-06_11:05:48.54190 golang.org/x/sync/errgroup.(*Group).Go.func1()
2022-12-06_11:05:48.54194 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67
2022-12-06_11:05:48.54198 created by golang.org/x/sync/errgroup.(*Group).Go
2022-12-06_11:05:48.54202 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d
2022-12-06_11:05:48.54207 
2022-12-06_11:05:48.54211 goroutine 134 [syscall, 20 minutes]:
2022-12-06_11:05:48.54215 os/signal.signal_recv()
2022-12-06_11:05:48.54218 	/usr/local/go/src/runtime/sigqueue.go:151 +0x2f
2022-12-06_11:05:48.54222 os/signal.loop()
2022-12-06_11:05:48.54226 	/usr/local/go/src/os/signal/signal_unix.go:23 +0x19
2022-12-06_11:05:48.54230 created by os/signal.Notify.func1.1
2022-12-06_11:05:48.54234 	/usr/local/go/src/os/signal/signal.go:151 +0x2a
2022-12-06_11:05:48.54238 
2022-12-06_11:05:48.54241 goroutine 154 [select, 20 minutes]:
2022-12-06_11:05:48.54245 main.listenToSystemSignals(0x0?)
2022-12-06_11:05:48.54249 	/var/cache/omnibus/src/grafana/pkg/cmd/grafana-server/main.go:190 +0x155
2022-12-06_11:05:48.54253 created by main.executeServer
2022-12-06_11:05:48.54257 	/var/cache/omnibus/src/grafana/pkg/cmd/grafana-server/main.go:159 +0x6dc
2022-12-06_11:05:48.54261 
2022-12-06_11:05:48.54265 goroutine 155 [IO wait]:
2022-12-06_11:05:48.54268 internal/poll.runtime_pollWait(0x7fce863f81d8, 0x72)
2022-12-06_11:05:48.54272 	/usr/local/go/src/runtime/netpoll.go:302 +0x89
2022-12-06_11:05:48.54276 internal/poll.(*pollDesc).wait(0xc00056a380?, 0xc00004ea00?, 0x0)
2022-12-06_11:05:48.54280 	/usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x32
2022-12-06_11:05:48.54283 internal/poll.(*pollDesc).waitRead(...)
2022-12-06_11:05:48.54287 	/usr/local/go/src/internal/poll/fd_poll_runtime.go:88
2022-12-06_11:05:48.54291 internal/poll.(*FD).Accept(0xc00056a380)
2022-12-06_11:05:48.54295 	/usr/local/go/src/internal/poll/fd_unix.go:614 +0x22c
2022-12-06_11:05:48.54298 net.(*netFD).accept(0xc00056a380)
2022-12-06_11:05:48.54304 	/usr/local/go/src/net/fd_unix.go:172 +0x35
2022-12-06_11:05:48.54308 net.(*TCPListener).accept(0xc000bd6240)
2022-12-06_11:05:48.54312 	/usr/local/go/src/net/tcpsock_posix.go:139 +0x28
2022-12-06_11:05:48.54315 net.(*TCPListener).Accept(0xc000bd6240)
2022-12-06_11:05:48.54320 	/usr/local/go/src/net/tcpsock.go:288 +0x3d
2022-12-06_11:05:48.54325 net/http.(*Server).Serve(0xc0008d60e0, {0x266f5a8, 0xc000bd6240})
2022-12-06_11:05:48.54329 	/usr/local/go/src/net/http/server.go:3039 +0x385
2022-12-06_11:05:48.54333 github.com/grafana/grafana/pkg/api.(*HTTPServer).Run(0xc000107b80, {0x2670b00?, 0xc000339940})
2022-12-06_11:05:48.54337 	/var/cache/omnibus/src/grafana/pkg/api/http_server.go:149 +0x72b
2022-12-06_11:05:48.54340 github.com/grafana/grafana/pkg/server.(*Server).Run.func1()
2022-12-06_11:05:48.54344 	/var/cache/omnibus/src/grafana/pkg/server/server.go:177 +0x49
2022-12-06_11:05:48.54348 golang.org/x/sync/errgroup.(*Group).Go.func1()
2022-12-06_11:05:48.54352 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67
2022-12-06_11:05:48.54356 created by golang.org/x/sync/errgroup.(*Group).Go
2022-12-06_11:05:48.54359 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d
2022-12-06_11:05:48.54365 
2022-12-06_11:05:48.54368 goroutine 156 [chan receive, 20 minutes]:
2022-12-06_11:05:48.54371 github.com/grafana/grafana/pkg/services/rendering.(*RenderingService).Run(0xc00020f180, {0x2670b00, 0xc000339940})
2022-12-06_11:05:48.54376 	/var/cache/omnibus/src/grafana/pkg/services/rendering/rendering.go:104 +0x1b9
2022-12-06_11:05:48.54379 github.com/grafana/grafana/pkg/server.(*Server).Run.func1()
2022-12-06_11:05:48.54382 	/var/cache/omnibus/src/grafana/pkg/server/server.go:177 +0x49
2022-12-06_11:05:48.54385 golang.org/x/sync/errgroup.(*Group).Go.func1()
2022-12-06_11:05:48.54388 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67
2022-12-06_11:05:48.54391 created by golang.org/x/sync/errgroup.(*Group).Go
2022-12-06_11:05:48.54394 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d
2022-12-06_11:05:48.54398 
2022-12-06_11:05:48.54401 goroutine 157 [chan receive, 20 minutes]:
2022-12-06_11:05:48.54405 github.com/grafana/grafana/pkg/plugins/backendplugin.(*manager).Run(0x0?, {0x2670b00, 0xc000339940})
2022-12-06_11:05:48.54409 	/var/cache/omnibus/src/grafana/pkg/plugins/backendplugin/manager.go:75 +0x3f
2022-12-06_11:05:48.54413 github.com/grafana/grafana/pkg/server.(*Server).Run.func1()
2022-12-06_11:05:48.54417 	/var/cache/omnibus/src/grafana/pkg/server/server.go:177 +0x49
2022-12-06_11:05:48.54421 golang.org/x/sync/errgroup.(*Group).Go.func1()
2022-12-06_11:05:48.54425 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67
2022-12-06_11:05:48.54430 created by golang.org/x/sync/errgroup.(*Group).Go
2022-12-06_11:05:48.54434 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d
2022-12-06_11:05:48.54438 
2022-12-06_11:05:48.54441 goroutine 158 [select, 2 minutes]:
2022-12-06_11:05:48.54444 github.com/grafana/grafana/pkg/infra/remotecache.(*databaseCache).Run(0x0?, {0x2670b00, 0xc000339940})
2022-12-06_11:05:48.54448 	/var/cache/omnibus/src/grafana/pkg/infra/remotecache/database_storage.go:32 +0x9f
2022-12-06_11:05:48.54451 github.com/grafana/grafana/pkg/infra/remotecache.(*RemoteCache).Run(0x0?, {0x2670b00, 0xc000339940})
2022-12-06_11:05:48.54455 	/var/cache/omnibus/src/grafana/pkg/infra/remotecache/remotecache.go:94 +0x58
2022-12-06_11:05:48.54459 github.com/grafana/grafana/pkg/server.(*Server).Run.func1()
2022-12-06_11:05:48.54463 	/var/cache/omnibus/src/grafana/pkg/server/server.go:177 +0x49
2022-12-06_11:05:48.54467 golang.org/x/sync/errgroup.(*Group).Go.func1()
2022-12-06_11:05:48.54471 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67
2022-12-06_11:05:48.54475 created by golang.org/x/sync/errgroup.(*Group).Go
2022-12-06_11:05:48.54479 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d
2022-12-06_11:05:48.54482 
2022-12-06_11:05:48.54486 goroutine 159 [select, 2 minutes]:
2022-12-06_11:05:48.54491 github.com/grafana/grafana/pkg/plugins.(*PluginManager).Run(0x35700000007?, {0x2670b00, 0xc000339940})
2022-12-06_11:05:48.54495 	/var/cache/omnibus/src/grafana/pkg/plugins/plugins.go:164 +0xc5
2022-12-06_11:05:48.54499 github.com/grafana/grafana/pkg/server.(*Server).Run.func1()
2022-12-06_11:05:48.54503 	/var/cache/omnibus/src/grafana/pkg/server/server.go:177 +0x49
2022-12-06_11:05:48.54507 golang.org/x/sync/errgroup.(*Group).Go.func1()
2022-12-06_11:05:48.54511 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67
2022-12-06_11:05:48.54515 created by golang.org/x/sync/errgroup.(*Group).Go
2022-12-06_11:05:48.54519 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d
2022-12-06_11:05:48.54523 
2022-12-06_11:05:48.54526 goroutine 160 [semacquire, 20 minutes]:
2022-12-06_11:05:48.54530 sync.runtime_Semacquire(0xc00044fa00?)
2022-12-06_11:05:48.54534 	/usr/local/go/src/runtime/sema.go:56 +0x25
2022-12-06_11:05:48.54538 sync.(*WaitGroup).Wait(0xc0000a3001?)
2022-12-06_11:05:48.54543 	/usr/local/go/src/sync/waitgroup.go:136 +0x52
2022-12-06_11:05:48.54547 golang.org/x/sync/errgroup.(*Group).Wait(0xc0000a3100)
2022-12-06_11:05:48.54550 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:40 +0x27
2022-12-06_11:05:48.54554 github.com/grafana/grafana/pkg/services/alerting.(*AlertEngine).Run(0xc0002565a0, {0x2670b00?, 0xc000339940?})
2022-12-06_11:05:48.54559 	/var/cache/omnibus/src/grafana/pkg/services/alerting/engine.go:66 +0x15f
2022-12-06_11:05:48.54563 github.com/grafana/grafana/pkg/server.(*Server).Run.func1()
2022-12-06_11:05:48.54566 	/var/cache/omnibus/src/grafana/pkg/server/server.go:177 +0x49
2022-12-06_11:05:48.54570 golang.org/x/sync/errgroup.(*Group).Go.func1()
2022-12-06_11:05:48.54574 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67
2022-12-06_11:05:48.54578 created by golang.org/x/sync/errgroup.(*Group).Go
2022-12-06_11:05:48.54584 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d
2022-12-06_11:05:48.54588 
2022-12-06_11:05:48.54592 goroutine 161 [select, 20 minutes]:
2022-12-06_11:05:48.54596 github.com/grafana/grafana/pkg/services/auth.(*UserAuthTokenService).Run(0xc000522d80, {0x2670b00, 0xc000339940})
2022-12-06_11:05:48.54600 	/var/cache/omnibus/src/grafana/pkg/services/auth/token_cleanup.go:25 +0x205
2022-12-06_11:05:48.54604 github.com/grafana/grafana/pkg/server.(*Server).Run.func1()
2022-12-06_11:05:48.54608 	/var/cache/omnibus/src/grafana/pkg/server/server.go:177 +0x49
2022-12-06_11:05:48.54611 golang.org/x/sync/errgroup.(*Group).Go.func1()
2022-12-06_11:05:48.54615 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67
2022-12-06_11:05:48.54621 created by golang.org/x/sync/errgroup.(*Group).Go
2022-12-06_11:05:48.54625 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d
2022-12-06_11:05:48.54628 
2022-12-06_11:05:48.54632 goroutine 162 [chan receive, 20 minutes]:
2022-12-06_11:05:48.54636 github.com/grafana/grafana/pkg/infra/metrics.(*InternalMetricsService).Run(0x0?, {0x2670b00, 0xc000339940})
2022-12-06_11:05:48.54640 	/var/cache/omnibus/src/grafana/pkg/infra/metrics/service.go:52 +0x170
2022-12-06_11:05:48.54644 github.com/grafana/grafana/pkg/server.(*Server).Run.func1()
2022-12-06_11:05:48.54648 	/var/cache/omnibus/src/grafana/pkg/server/server.go:177 +0x49
2022-12-06_11:05:48.54652 golang.org/x/sync/errgroup.(*Group).Go.func1()
2022-12-06_11:05:48.54655 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67
2022-12-06_11:05:48.54659 created by golang.org/x/sync/errgroup.(*Group).Go
2022-12-06_11:05:48.54663 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d
2022-12-06_11:05:48.54667 
2022-12-06_11:05:48.54671 goroutine 163 [select, 20 minutes]:
2022-12-06_11:05:48.54675 github.com/grafana/grafana/pkg/services/notifications.(*NotificationService).Run(0xc000249e00, {0x2670b00, 0xc000339940})
2022-12-06_11:05:48.54679 	/var/cache/omnibus/src/grafana/pkg/services/notifications/notifications.go:78 +0x118
2022-12-06_11:05:48.54684 github.com/grafana/grafana/pkg/server.(*Server).Run.func1()
2022-12-06_11:05:48.54688 	/var/cache/omnibus/src/grafana/pkg/server/server.go:177 +0x49
2022-12-06_11:05:48.54691 golang.org/x/sync/errgroup.(*Group).Go.func1()
2022-12-06_11:05:48.54695 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67
2022-12-06_11:05:48.54699 created by golang.org/x/sync/errgroup.(*Group).Go
2022-12-06_11:05:48.54703 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d
2022-12-06_11:05:48.54707 
2022-12-06_11:05:48.54711 goroutine 164 [chan receive, 20 minutes]:
2022-12-06_11:05:48.54715 github.com/grafana/grafana/pkg/infra/tracing.(*TracingService).Run(0xc000204a80, {0x2670b00?, 0xc000339940?})
2022-12-06_11:05:48.54721 	/var/cache/omnibus/src/grafana/pkg/infra/tracing/tracing.go:129 +0x37
2022-12-06_11:05:48.54726 github.com/grafana/grafana/pkg/server.(*Server).Run.func1()
2022-12-06_11:05:48.54730 	/var/cache/omnibus/src/grafana/pkg/server/server.go:177 +0x49
2022-12-06_11:05:48.54733 golang.org/x/sync/errgroup.(*Group).Go.func1()
2022-12-06_11:05:48.54737 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67
2022-12-06_11:05:48.54742 created by golang.org/x/sync/errgroup.(*Group).Go
2022-12-06_11:05:48.54748 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d
2022-12-06_11:05:48.54752 
2022-12-06_11:05:48.54756 goroutine 165 [select, 2 minutes]:
2022-12-06_11:05:48.54760 github.com/grafana/grafana/pkg/services/cleanup.(*CleanUpService).Run(0xc0004d82d0, {0x2670b00, 0xc000339940})
2022-12-06_11:05:48.54764 	/var/cache/omnibus/src/grafana/pkg/services/cleanup/cleanup.go:42 +0xca
2022-12-06_11:05:48.54767 github.com/grafana/grafana/pkg/server.(*Server).Run.func1()
2022-12-06_11:05:48.54771 	/var/cache/omnibus/src/grafana/pkg/server/server.go:177 +0x49
2022-12-06_11:05:48.54775 golang.org/x/sync/errgroup.(*Group).Go.func1()
2022-12-06_11:05:48.54779 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67
2022-12-06_11:05:48.54783 created by golang.org/x/sync/errgroup.(*Group).Go
2022-12-06_11:05:48.54787 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d
2022-12-06_11:05:48.54791 
2022-12-06_11:05:48.54794 goroutine 166 [select, 20 minutes]:
2022-12-06_11:05:48.54798 github.com/grafana/grafana/pkg/infra/usagestats.(*UsageStatsService).Run(0x0?, {0x2670b00, 0xc000339940})
2022-12-06_11:05:48.54802 	/var/cache/omnibus/src/grafana/pkg/infra/usagestats/service.go:63 +0x185
2022-12-06_11:05:48.54806 github.com/grafana/grafana/pkg/server.(*Server).Run.func1()
2022-12-06_11:05:48.54811 	/var/cache/omnibus/src/grafana/pkg/server/server.go:177 +0x49
2022-12-06_11:05:48.54815 golang.org/x/sync/errgroup.(*Group).Go.func1()
2022-12-06_11:05:48.54819 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67
2022-12-06_11:05:48.54823 created by golang.org/x/sync/errgroup.(*Group).Go
2022-12-06_11:05:48.54827 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d
2022-12-06_11:05:48.54831 
2022-12-06_11:05:48.54834 goroutine 167 [select, 20 minutes]:
2022-12-06_11:05:48.54838 github.com/grafana/grafana/pkg/services/provisioning.(*provisioningServiceImpl).Run(0xc0003a2600, {0x2670b00, 0xc000339940})
2022-12-06_11:05:48.54842 	/var/cache/omnibus/src/grafana/pkg/services/provisioning/provisioning.go:105 +0x212
2022-12-06_11:05:48.54846 github.com/grafana/grafana/pkg/server.(*Server).Run.func1()
2022-12-06_11:05:48.54850 	/var/cache/omnibus/src/grafana/pkg/server/server.go:177 +0x49
2022-12-06_11:05:48.54854 golang.org/x/sync/errgroup.(*Group).Go.func1()
2022-12-06_11:05:48.54857 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67
2022-12-06_11:05:48.54861 created by golang.org/x/sync/errgroup.(*Group).Go
2022-12-06_11:05:48.54865 	/root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d
2022-12-06_11:05:48.54870 
2022-12-06_11:05:48.54874 goroutine 110 [select]:
2022-12-06_11:05:48.54878 github.com/grafana/grafana/pkg/plugins/backendplugin.restartKilledProcess({0x2670b00, 0xc000339940}, {0x2678f90, 0xc00020f0e0})
2022-12-06_11:05:48.54882 	/var/cache/omnibus/src/grafana/pkg/plugins/backendplugin/manager.go:442 +0xad
2022-12-06_11:05:48.54886 github.com/grafana/grafana/pkg/plugins/backendplugin.startPluginAndRestartKilledProcesses.func1({0x2670b00?, 0xc000339940?}, {0x2678f90?, 0xc00020f0e0?})
2022-12-06_11:05:48.54890 	/var/cache/omnibus/src/grafana/pkg/plugins/backendplugin/manager.go:430 +0x3b
2022-12-06_11:05:48.54894 created by github.com/grafana/grafana/pkg/plugins/backendplugin.startPluginAndRestartKilledProcesses
2022-12-06_11:05:48.54899 	/var/cache/omnibus/src/grafana/pkg/plugins/backendplugin/manager.go:429 +0xcb
2022-12-06_11:05:48.54903 
2022-12-06_11:05:48.54907 goroutine 47 [chan receive, 20 minutes]:
2022-12-06_11:05:48.54911 github.com/grafana/grafana/pkg/api.(*HTTPServer).Run.func1()
2022-12-06_11:05:48.54914 	/var/cache/omnibus/src/grafana/pkg/api/http_server.go:141 +0x71
2022-12-06_11:05:48.54918 created by github.com/grafana/grafana/pkg/api.(*HTTPServer).Run
2022-12-06_11:05:48.54922 	/var/cache/omnibus/src/grafana/pkg/api/http_server.go:138 +0x5be
2022-12-06_11:05:48.54926 
2022-12-06_11:05:48.54930 goroutine 168 [select, 2 minutes]:
2022-12-06_11:05:48.54934 github.com/grafana/grafana/pkg/services/provisioning/dashboards.(*FileReader).pollChanges(0xc0003da3c0, {0x2670b00, 0xc0003db6c0})
2022-12-06_11:05:48.54939 	/var/cache/omnibus/src/grafana/pkg/services/provisioning/dashboards/file_reader.go:68 +0xa9
2022-12-06_11:05:48.54943 created by github.com/grafana/grafana/pkg/services/provisioning/dashboards.(*Provisioner).PollChanges
2022-12-06_11:05:48.54947 	/var/cache/omnibus/src/grafana/pkg/services/provisioning/dashboards/dashboard.go:93 +0x45
2022-12-06_11:05:48.54950 
2022-12-06_11:05:48.54954 goroutine 265 [IO wait]:
2022-12-06_11:05:48.54958 internal/poll.runtime_pollWait(0x7fce863f80e8, 0x72)
2022-12-06_11:05:48.54962 	/usr/local/go/src/runtime/netpoll.go:302 +0x89
2022-12-06_11:05:48.54966 internal/poll.(*pollDesc).wait(0xc00056a580?, 0xc000527980?, 0x0)
2022-12-06_11:05:48.54969 	/usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x32
2022-12-06_11:05:48.54973 internal/poll.(*pollDesc).waitRead(...)
2022-12-06_11:05:48.54977 	/usr/local/go/src/internal/poll/fd_poll_runtime.go:88
2022-12-06_11:05:48.54982 internal/poll.(*FD).Read(0xc00056a580, {0xc000527980, 0x18c9, 0x18c9})
2022-12-06_11:05:48.54987 	/usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a
2022-12-06_11:05:48.54990 net.(*netFD).Read(0xc00056a580, {0xc000527980?, 0xc000527985?, 0x1e0?})
2022-12-06_11:05:48.54994 	/usr/local/go/src/net/fd_posix.go:55 +0x29
2022-12-06_11:05:48.54998 net.(*conn).Read(0xc0000c4d08, {0xc000527980?, 0xc000577820?, 0xc000f37318?})
2022-12-06_11:05:48.55003 	/usr/local/go/src/net/net.go:183 +0x45
2022-12-06_11:05:48.55007 crypto/tls.(*atLeastReader).Read(0xc000f7c768, {0xc000527980?, 0x0?, 0x21aeb00?})
2022-12-06_11:05:48.55011 	/usr/local/go/src/crypto/tls/conn.go:785 +0x3d
2022-12-06_11:05:48.55015 bytes.(*Buffer).ReadFrom(0xc000f373f8, {0x2657ee0, 0xc000f7c768})
2022-12-06_11:05:48.55019 	/usr/local/go/src/bytes/buffer.go:204 +0x98
2022-12-06_11:05:48.55023 crypto/tls.(*Conn).readFromUntil(0xc000f37180, {0x7fce84fcee60?, 0xc0003dac00}, 0x10c?)
2022-12-06_11:05:48.55027 	/usr/local/go/src/crypto/tls/conn.go:807 +0xe5
2022-12-06_11:05:48.55030 crypto/tls.(*Conn).readRecordOrCCS(0xc000f37180, 0x0)
2022-12-06_11:05:48.55034 	/usr/local/go/src/crypto/tls/conn.go:614 +0x116
2022-12-06_11:05:48.55038 crypto/tls.(*Conn).readRecord(...)
2022-12-06_11:05:48.55042 	/usr/local/go/src/crypto/tls/conn.go:582
2022-12-06_11:05:48.55046 crypto/tls.(*Conn).Read(0xc000f37180, {0xc000725000, 0x1000, 0x1?})
2022-12-06_11:05:48.55050 	/usr/local/go/src/crypto/tls/conn.go:1285 +0x16f
2022-12-06_11:05:48.55053 net/http.(*persistConn).Read(0xc000223320, {0xc000725000?, 0xc0000a83c0?, 0xc000577d30?})
2022-12-06_11:05:48.55057 	/usr/local/go/src/net/http/transport.go:1929 +0x4e
2022-12-06_11:05:48.55061 bufio.(*Reader).fill(0xc000bd1c20)
2022-12-06_11:05:48.55065 	/usr/local/go/src/bufio/bufio.go:106 +0x103
2022-12-06_11:05:48.55069 bufio.(*Reader).Peek(0xc000bd1c20, 0x1)
2022-12-06_11:05:48.55072 	/usr/local/go/src/bufio/bufio.go:144 +0x5d
2022-12-06_11:05:48.55078 net/http.(*persistConn).readLoop(0xc000223320)
2022-12-06_11:05:48.55081 	/usr/local/go/src/net/http/transport.go:2093 +0x1ac
2022-12-06_11:05:48.55085 created by net/http.(*Transport).dialConn
2022-12-06_11:05:48.55089 	/usr/local/go/src/net/http/transport.go:1750 +0x173e
2022-12-06_11:05:48.55093 
2022-12-06_11:05:48.55098 goroutine 239 [IO wait]:
2022-12-06_11:05:48.55102 internal/poll.runtime_pollWait(0x7fce863f7f08, 0x72)
2022-12-06_11:05:48.55106 	/usr/local/go/src/runtime/netpoll.go:302 +0x89
2022-12-06_11:05:48.55110 internal/poll.(*pollDesc).wait(0xc0003fc100?, 0xc000d6a5e1?, 0x0)
2022-12-06_11:05:48.55114 	/usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x32
2022-12-06_11:05:48.55117 internal/poll.(*pollDesc).waitRead(...)
2022-12-06_11:05:48.55123 	/usr/local/go/src/internal/poll/fd_poll_runtime.go:88
2022-12-06_11:05:48.55127 internal/poll.(*FD).Read(0xc0003fc100, {0xc000d6a5e1, 0x1, 0x1})
2022-12-06_11:05:48.55131 	/usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a
2022-12-06_11:05:48.55135 net.(*netFD).Read(0xc0003fc100, {0xc000d6a5e1?, 0xc0008cfba0?, 0xc0008cfc28?})
2022-12-06_11:05:48.55138 	/usr/local/go/src/net/fd_posix.go:55 +0x29
2022-12-06_11:05:48.55142 net.(*conn).Read(0xc0000c46e8, {0xc000d6a5e1?, 0x0?, 0xc00053af01?})
2022-12-06_11:05:48.55146 	/usr/local/go/src/net/net.go:183 +0x45
2022-12-06_11:05:48.55150 net/http.(*connReader).backgroundRead(0xc000d6a5d0)
2022-12-06_11:05:48.55171 	/usr/local/go/src/net/http/server.go:672 +0x3f
2022-12-06_11:05:48.55175 created by net/http.(*connReader).startBackgroundRead
2022-12-06_11:05:48.55180 	/usr/local/go/src/net/http/server.go:668 +0xca
2022-12-06_11:05:48.55184 
2022-12-06_11:05:48.55188 goroutine 175 [chan receive, 4 minutes]:
2022-12-06_11:05:48.55192 github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init.func1()
2022-12-06_11:05:48.55195 	/var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:168 +0x34
2022-12-06_11:05:48.55199 created by github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init
2022-12-06_11:05:48.55202 	/var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:166 +0x65
2022-12-06_11:05:48.55205 
2022-12-06_11:05:48.55208 goroutine 176 [chan receive, 4 minutes]:
2022-12-06_11:05:48.55211 github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init.func1()
2022-12-06_11:05:48.55215 	/var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:168 +0x34
2022-12-06_11:05:48.55218 created by github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init
2022-12-06_11:05:48.55221 	/var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:166 +0x65
2022-12-06_11:05:48.55224 
2022-12-06_11:05:48.55227 goroutine 177 [chan receive, 4 minutes]:
2022-12-06_11:05:48.55230 github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init.func1()
2022-12-06_11:05:48.55233 	/var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:168 +0x34
2022-12-06_11:05:48.55237 created by github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init
2022-12-06_11:05:48.55240 	/var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:166 +0x65
2022-12-06_11:05:48.55245 
2022-12-06_11:05:48.55248 goroutine 194 [chan receive, 4 minutes]:
2022-12-06_11:05:48.55252 github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init.func1()
2022-12-06_11:05:48.55256 	/var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:168 +0x34
2022-12-06_11:05:48.55260 created by github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init
2022-12-06_11:05:48.55264 	/var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:166 +0x65
2022-12-06_11:05:48.55268 
2022-12-06_11:05:48.55271 goroutine 195 [chan receive, 4 minutes]:
2022-12-06_11:05:48.55275 github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init.func1()
2022-12-06_11:05:48.55279 	/var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:168 +0x34
2022-12-06_11:05:48.55283 created by github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init
2022-12-06_11:05:48.55287 	/var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:166 +0x65
2022-12-06_11:05:48.55291 
2022-12-06_11:05:48.55294 goroutine 196 [chan receive, 4 minutes]:
2022-12-06_11:05:48.55298 github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init.func1()
2022-12-06_11:05:48.55302 	/var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:168 +0x34
2022-12-06_11:05:48.55306 created by github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init
2022-12-06_11:05:48.55310 	/var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:166 +0x65
2022-12-06_11:05:48.55315 
2022-12-06_11:05:48.55319 goroutine 197 [chan receive, 4 minutes]:
2022-12-06_11:05:48.55322 github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init.func1()
2022-12-06_11:05:48.55328 	/var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:168 +0x34
2022-12-06_11:05:48.55332 created by github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init
2022-12-06_11:05:48.55335 	/var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:166 +0x65
2022-12-06_11:05:48.55339 
2022-12-06_11:05:48.55343 goroutine 198 [chan receive, 4 minutes]:
2022-12-06_11:05:48.55347 github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init.func1()
2022-12-06_11:05:48.55351 	/var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:168 +0x34
2022-12-06_11:05:48.55357 created by github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init
2022-12-06_11:05:48.55362 	/var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:166 +0x65
2022-12-06_11:05:48.55366 
2022-12-06_11:05:48.55369 goroutine 199 [chan receive, 4 minutes]:
2022-12-06_11:05:48.55373 github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init.func1()
2022-12-06_11:05:48.55377 	/var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:168 +0x34
2022-12-06_11:05:48.55382 created by github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init
2022-12-06_11:05:48.55387 	/var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:166 +0x65
2022-12-06_11:05:48.55391 
2022-12-06_11:05:48.55395 goroutine 200 [chan receive, 4 minutes]:
2022-12-06_11:05:48.55399 github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init.func1()
2022-12-06_11:05:48.55403 	/var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:168 +0x34
2022-12-06_11:05:48.55408 created by github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init
2022-12-06_11:05:48.55412 	/var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:166 +0x65
2022-12-06_11:05:48.55416 
2022-12-06_11:05:48.55419 goroutine 266 [select]:
2022-12-06_11:05:48.55423 net/http.(*persistConn).writeLoop(0xc000223320)
2022-12-06_11:05:48.55427 	/usr/local/go/src/net/http/transport.go:2392 +0xf5
2022-12-06_11:05:48.55431 created by net/http.(*Transport).dialConn
2022-12-06_11:05:48.55434 	/usr/local/go/src/net/http/transport.go:1751 +0x1791
2022-12-06_11:05:48.60991 t=2022-12-06T12:05:48+0100 lvl=info msg="Starting Grafana" logger=server version=7.5.16 commit=c0e2ad126c branch=HEAD compiled=2022-05-19T12:39:41+0200
2022-12-06_11:05:48.61003 t=2022-12-06T12:05:48+0100 lvl=info msg="Config loaded from" logger=settings file=/var/opt/gitlab/grafana/conf/defaults.ini
2022-12-06_11:05:48.61007 t=2022-12-06T12:05:48+0100 lvl=info msg="Config loaded from" logger=settings file=/var/opt/gitlab/grafana/grafana.ini
2022-12-06_11:05:48.61014 t=2022-12-06T12:05:48+0100 lvl=info msg="Path Home" logger=settings path=/var/opt/gitlab/grafana
2022-12-06_11:05:48.61017 t=2022-12-06T12:05:48+0100 lvl=info msg="Path Data" logger=settings path=/var/opt/gitlab/grafana/data
2022-12-06_11:05:48.61022 t=2022-12-06T12:05:48+0100 lvl=info msg="Path Logs" logger=settings path=/var/log/gitlab/grafana
2022-12-06_11:05:48.61027 t=2022-12-06T12:05:48+0100 lvl=info msg="Path Plugins" logger=settings path=/var/opt/gitlab/grafana/data/plugins
2022-12-06_11:05:48.61032 t=2022-12-06T12:05:48+0100 lvl=info msg="Path Provisioning" logger=settings path=/var/opt/gitlab/grafana/provisioning
2022-12-06_11:05:48.61037 t=2022-12-06T12:05:48+0100 lvl=info msg="App mode production" logger=settings
2022-12-06_11:05:48.61195 t=2022-12-06T12:05:48+0100 lvl=info msg="Connecting to DB" logger=sqlstore dbtype=sqlite3
2022-12-06_11:05:48.61315 t=2022-12-06T12:05:48+0100 lvl=info msg="Starting DB migrations" logger=migrator
2022-12-06_11:05:48.62625 t=2022-12-06T12:05:48+0100 lvl=info msg="migrations completed" logger=migrator performed=0 skipped=279 duration=1.490749ms
2022-12-06_11:05:48.62919 t=2022-12-06T12:05:48+0100 lvl=info msg="Starting plugin search" logger=plugins
2022-12-06_11:05:48.69082 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.690741+01:00","queryType":"exponential_heatmap_bucket_data"}
2022-12-06_11:05:48.69100 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.690971+01:00","queryType":"linear_heatmap_bucket_data"}
2022-12-06_11:05:48.69114 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.691086+01:00","queryType":"random_walk"}
2022-12-06_11:05:48.69131 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.691282+01:00","queryType":"predictable_pulse"}
2022-12-06_11:05:48.69141 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.691385+01:00","queryType":"predictable_csv_wave"}
2022-12-06_11:05:48.69151 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.691484+01:00","queryType":"random_walk_table"}
2022-12-06_11:05:48.69161 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.691587+01:00","queryType":"slow_query"}
2022-12-06_11:05:48.69170 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.691677+01:00","queryType":"no_data_points"}
2022-12-06_11:05:48.69180 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.691777+01:00","queryType":"datapoints_outside_range"}
2022-12-06_11:05:48.69190 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.691874+01:00","queryType":"manual_entry"}
2022-12-06_11:05:48.69199 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.691965+01:00","queryType":"csv_metric_values"}
2022-12-06_11:05:48.69208 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.692057+01:00","queryType":"streaming_client"}
2022-12-06_11:05:48.69217 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.692148+01:00","queryType":"live"}
2022-12-06_11:05:48.69227 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.692243+01:00","queryType":"grafana_api"}
2022-12-06_11:05:48.69236 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.692337+01:00","queryType":"arrow"}
2022-12-06_11:05:48.69245 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.692428+01:00","queryType":"annotations"}
2022-12-06_11:05:48.69254 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.692521+01:00","queryType":"table_static"}
2022-12-06_11:05:48.69266 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.692612+01:00","queryType":"random_walk_with_error"}
2022-12-06_11:05:48.69275 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.692726+01:00","queryType":"server_error_500"}
2022-12-06_11:05:48.69284 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.692819+01:00","queryType":"logs"}
2022-12-06_11:05:48.69296 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.692909+01:00","queryType":"node_graph"}
2022-12-06_11:05:48.69305 {"@level":"debug","@message":"datasource: registering query type fallback handler","@timestamp":"2022-12-06T12:05:48.693025+01:00"}
2022-12-06_11:05:48.70735 t=2022-12-06T12:05:48+0100 lvl=info msg="HTTP Server Listen" logger=http.server address=127.0.0.1:3000 protocol=http subUrl=/-/grafana socket=

Results of GitLab environment info

Expand for output related to GitLab environment info 
System information
System:		Debian 11
Current User:	git
Using RVM:	no
Ruby Version:	2.7.6p219
Gem Version:	3.1.6
Bundler Version:2.3.15
Rake Version:	13.0.6
Redis Version:	6.2.7
Sidekiq Version:6.5.7
Go Version:	unknown

GitLab information
Version:	15.6.2
Revision:	2d7e4701985
Directory:	/opt/gitlab/embedded/service/gitlab-rails
DB Adapter:	PostgreSQL
DB Version:	12.12
URL:		https://gitlab.domain.redacted.org
HTTP Clone URL:	https://gitlab.domain.redacted.org/some-group/some-project.git
SSH Clone URL:	git@gitlab.domain.redacted.org:some-group/some-project.git
Using LDAP:	no
Using Omniauth:	yes
Omniauth Providers: google_oauth2

GitLab Shell
Version:	14.13.0
Repository storage paths:
- default: 	/srv/gitlab/git-data/repositories
GitLab Shell path:		/opt/gitlab/embedded/service/gitlab-shell

Results of GitLab application Check

Expand for output related to the GitLab application check 
Checking GitLab subtasks ...

Checking GitLab Shell ...


GitLab Shell: ... GitLab Shell version >= 14.13.0 ? ... OK (14.13.0)
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Internal API available: OK
Redis available via internal API: OK
gitlab-shell self-check successful


Checking GitLab Shell ... Finished


Checking Gitaly ...


Gitaly: ... default ... OK


Checking Gitaly ... Finished


Checking Sidekiq ...


Sidekiq: ... Running? ... yes
Number of Sidekiq processes (cluster/worker) ... 1/1


Checking Sidekiq ... Finished


Checking Incoming Email ...


Incoming Email: ... Reply by email is disabled in config/gitlab.yml


Checking Incoming Email ... Finished


Checking LDAP ...


LDAP: ... LDAP is disabled in config/gitlab.yml


Checking LDAP ... Finished


Checking GitLab App ...


Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Systemd unit files or init script exist? ... skipped (omnibus-gitlab has neither init script nor systemd units)
Systemd unit files or init script up-to-date? ... skipped (omnibus-gitlab has neither init script nor systemd units)
Projects have namespace: ...
3/1 ... yes
3/2 ... yes
4/3 ... yes
4/4 ... yes
8/5 ... yes
4/6 ... yes
10/7 ... yes
7/8 ... yes
12/13 ... yes
7/15 ... yes
12/16 ... yes
12/17 ... yes
10/18 ... yes
10/20 ... yes
22/21 ... yes
15/22 ... yes
3/23 ... yes
23/24 ... yes
10/25 ... yes
10/26 ... yes
10/27 ... yes
10/28 ... yes
3/29 ... yes
23/30 ... yes
23/31 ... yes
27/36 ... yes
23/37 ... yes
23/38 ... yes
23/39 ... yes
10/40 ... yes
23/42 ... yes
10/43 ... yes
8/45 ... yes
10/46 ... yes
10/47 ... yes
10/48 ... yes
22/50 ... yes
87/52 ... yes
87/53 ... yes
87/54 ... yes
87/55 ... yes
87/56 ... yes
87/57 ... yes
4/58 ... yes
34/59 ... yes
22/60 ... yes
23/61 ... yes
35/62 ... yes
35/63 ... yes
35/64 ... yes
35/65 ... yes
35/66 ... yes
35/67 ... yes
35/68 ... yes
8/69 ... yes
8/70 ... yes
23/71 ... yes
23/72 ... yes
22/73 ... yes
38/74 ... yes
38/75 ... yes
38/76 ... yes
38/77 ... yes
38/78 ... yes
35/79 ... yes
9/80 ... yes
23/81 ... yes
23/83 ... yes
23/84 ... yes
23/85 ... yes
10/86 ... yes
36/87 ... yes
12/88 ... yes
87/89 ... yes
87/90 ... yes
22/91 ... yes
35/92 ... yes
22/93 ... yes
22/94 ... yes
22/96 ... yes
22/97 ... yes
10/98 ... yes
35/99 ... yes
38/100 ... yes
7/101 ... yes
10/102 ... yes
27/103 ... yes
23/104 ... yes
23/105 ... yes
22/106 ... yes
10/107 ... yes
86/108 ... yes
86/109 ... yes
86/110 ... yes
86/111 ... yes
86/112 ... yes
86/113 ... yes
86/115 ... yes
34/116 ... yes
86/117 ... yes
23/118 ... yes
23/119 ... yes
45/120 ... yes
4/121 ... yes
45/122 ... yes
2/123 ... yes
45/124 ... yes
45/125 ... yes
45/126 ... yes
2/127 ... yes
23/128 ... yes
8/129 ... yes
22/130 ... yes
48/133 ... yes
48/134 ... yes
50/138 ... yes
45/139 ... yes
48/140 ... yes
12/142 ... yes
45/143 ... yes
7/144 ... yes
35/146 ... yes
35/147 ... yes
35/148 ... yes
9/157 ... yes
9/161 ... yes
87/163 ... yes
4/164 ... yes
23/166 ... yes
23/167 ... yes
10/168 ... yes
10/169 ... yes
23/170 ... yes
8/171 ... yes
8/172 ... yes
8/173 ... yes
8/174 ... yes
34/175 ... yes
31/176 ... yes
23/177 ... yes
45/178 ... yes
45/179 ... yes
45/180 ... yes
34/181 ... yes
56/182 ... yes
57/183 ... yes
86/185 ... yes
57/186 ... yes
45/187 ... yes
23/188 ... yes
48/189 ... yes
57/190 ... yes
60/191 ... yes
50/192 ... yes
60/193 ... yes
23/194 ... yes
57/195 ... yes
60/196 ... yes
60/197 ... yes
8/198 ... yes
56/199 ... yes
22/200 ... yes
23/201 ... yes
61/202 ... yes
45/203 ... yes
12/204 ... yes
45/205 ... yes
45/206 ... yes
23/207 ... yes
23/208 ... yes
23/209 ... yes
23/210 ... yes
87/211 ... yes
55/212 ... yes
38/213 ... yes
39/214 ... yes
2/216 ... yes
75/217 ... yes
35/218 ... yes
87/219 ... yes
56/220 ... yes
27/221 ... yes
39/222 ... yes
45/223 ... yes
45/224 ... yes
87/225 ... yes
35/226 ... yes
35/227 ... yes
35/228 ... yes
70/229 ... yes
64/230 ... yes
64/231 ... yes
70/233 ... yes
61/234 ... yes
23/235 ... yes
86/236 ... yes
64/237 ... yes
39/238 ... yes
70/239 ... yes
23/240 ... yes
35/241 ... yes
35/242 ... yes
70/243 ... yes
39/244 ... yes
57/245 ... yes
23/246 ... yes
23/247 ... yes
23/248 ... yes
70/250 ... yes
70/251 ... yes
35/252 ... yes
39/253 ... yes
39/255 ... yes
61/256 ... yes
61/257 ... yes
71/258 ... yes
87/259 ... yes
72/260 ... yes
61/261 ... yes
70/262 ... yes
55/263 ... yes
48/264 ... yes
38/265 ... yes
48/266 ... yes
76/267 ... yes
34/268 ... yes
76/269 ... yes
70/270 ... yes
75/271 ... yes
48/273 ... yes
48/274 ... yes
23/275 ... yes
38/276 ... yes
23/277 ... yes
45/278 ... yes
75/279 ... yes
75/280 ... yes
76/281 ... yes
76/282 ... yes
76/283 ... yes
76/284 ... yes
10/285 ... yes
38/286 ... yes
413/304 ... yes
76/305 ... yes
76/306 ... yes
55/307 ... yes
10/308 ... yes
60/313 ... yes
57/314 ... yes
57/315 ... yes
70/316 ... yes
76/317 ... yes
76/318 ... yes
413/319 ... yes
23/320 ... yes
56/321 ... yes
39/322 ... yes
76/323 ... yes
70/324 ... yes
55/325 ... yes
2/326 ... yes
2/327 ... yes
2/328 ... yes
2/329 ... yes
45/330 ... yes
10/331 ... yes
84/332 ... yes
86/333 ... yes
7/334 ... yes
45/335 ... yes
76/336 ... yes
55/337 ... yes
48/338 ... yes
87/339 ... yes
413/340 ... yes
59/341 ... yes
70/342 ... yes
75/343 ... yes
55/344 ... yes
39/345 ... yes
45/346 ... yes
35/347 ... yes
70/348 ... yes
86/349 ... yes
76/350 ... yes
23/351 ... yes
2/352 ... yes
27/353 ... yes
55/354 ... yes
70/355 ... yes
48/356 ... yes
70/357 ... yes
70/358 ... yes
76/359 ... yes
34/360 ... yes
2/361 ... yes
8/362 ... yes
2/363 ... yes
23/364 ... yes
23/365 ... yes
408/366 ... yes
76/367 ... yes
35/368 ... yes
418/369 ... yes
61/370 ... yes
4/371 ... yes
45/372 ... yes
50/373 ... yes
50/374 ... yes
50/375 ... yes
23/376 ... yes
38/377 ... yes
38/378 ... yes
23/379 ... yes
22/380 ... yes
38/381 ... yes
75/382 ... yes
45/383 ... yes
45/384 ... yes
45/385 ... yes
75/386 ... yes
76/387 ... yes
35/388 ... yes
35/389 ... yes
70/390 ... yes
8/391 ... yes
8/392 ... yes
12/393 ... yes
76/394 ... yes
23/395 ... yes
76/396 ... yes
2/397 ... yes
2/398 ... yes
76/400 ... yes
46/401 ... yes
87/402 ... yes
456/403 ... yes
449/404 ... yes
70/405 ... yes
449/406 ... yes
12/408 ... yes
10/409 ... yes
70/410 ... yes
76/411 ... yes
76/412 ... yes
38/413 ... yes
84/414 ... yes
84/415 ... yes
70/416 ... yes
27/417 ... yes
84/418 ... yes
70/419 ... yes
86/420 ... yes
76/421 ... yes
76/422 ... yes
75/423 ... yes
485/424 ... yes
12/425 ... yes
489/426 ... yes
87/427 ... yes
87/428 ... yes
87/429 ... yes
50/430 ... yes
50/431 ... yes
48/432 ... yes
48/433 ... yes
48/434 ... yes
48/435 ... yes
70/436 ... yes
76/437 ... yes
35/438 ... yes
39/439 ... yes
87/440 ... yes
60/441 ... yes
60/442 ... yes
76/443 ... yes
76/444 ... yes
75/445 ... yes
45/446 ... yes
522/447 ... yes
35/448 ... yes
76/449 ... yes
70/450 ... yes
39/451 ... yes
70/452 ... yes
38/453 ... yes
76/454 ... yes
87/455 ... yes
70/456 ... yes
87/457 ... yes
23/458 ... yes
76/459 ... yes
86/461 ... yes
81/462 ... yes
Redis version >= 6.0.0? ... yes
Ruby version >= 2.7.2 ? ... yes (2.7.6)
Git user has default SSH configuration? ... yes
Active users: ... 52
Is authorized keys file accessible? ... yes
GitLab configured to store new projects in hashed storage? ... yes
All projects are in hashed storage? ... yes


Checking GitLab App ... Finished


Checking GitLab subtasks ... Finished

Additional info

InfluxDB instance I am attempting to use is in local network and the connection has been functional prior to the update. When creating a temporary instance of Grafana in a Docker container the connection can be established without issues, including when creating the container on the same machine that GitLab instance is running. I am eager to provide any additional information should it be required.