Grafana crashing when using external InfluxDB data source
Summary
Grafana provided with GitLab Omnibus crashes when attempting to connect to external InfluxDB data source after GitLab upgrade to version 15.6
Steps to reproduce
- Navigate to
gitlab.example.com/-/grafana
, log in, go to data sources configuration, - Add new InfluxDB data source,
- Specify InfluxDB data source configuration:
- Query language: Flux
- URL:
https://influxdb.example.com:8086
- Basic auth,
- Correct organization,
- Valid token,
- Valid default bucket
- Click Save & Test.
What is the current bug behavior?
After clicking Save & Test UI displays two messages: Data source updated and Error reading InfluxDB. Grafana logs report SIGSEGV
and the service is restarted.
What is the expected correct behavior?
After clicking Save & Test UI displays two messages: Data source updated and x buckets found. Logs are clean and newly created data source can be used in dashboards.
Relevant logs and/or screenshots
Grafana log immediately after clicking Save & Test:
2022-12-06_11:05:48.53124 unexpected fault address 0x0 2022-12-06_11:05:48.53145 fatal error: fault 2022-12-06_11:05:48.53342 [signal SIGSEGV: segmentation violation code=0x80 addr=0x0 pc=0x46f09f] 2022-12-06_11:05:48.53349 2022-12-06_11:05:48.53352 goroutine 238 [running]: 2022-12-06_11:05:48.53356 runtime.throw({0x2293bd3?, 0x18?}) 2022-12-06_11:05:48.53362 /usr/local/go/src/runtime/panic.go:992 +0x71 fp=0xc0008306f0 sp=0xc0008306c0 pc=0x43eb11 2022-12-06_11:05:48.53367 runtime.sigpanic() 2022-12-06_11:05:48.53370 /usr/local/go/src/runtime/signal_unix.go:825 +0x2ec fp=0xc000830740 sp=0xc0008306f0 pc=0x454d0c 2022-12-06_11:05:48.53374 aeshashbody() 2022-12-06_11:05:48.53378 /usr/local/go/src/runtime/asm_amd64.s:1343 +0x39f fp=0xc000830748 sp=0xc000830740 pc=0x46f09f 2022-12-06_11:05:48.53384 runtime.mapiternext(0xc0009c4500) 2022-12-06_11:05:48.53388 /usr/local/go/src/runtime/map.go:934 +0x2cb fp=0xc0008307b8 sp=0xc000830748 pc=0x41890b 2022-12-06_11:05:48.53392 runtime.mapiterinit(0x0?, 0x8?, 0x1?) 2022-12-06_11:05:48.53397 /usr/local/go/src/runtime/map.go:861 +0x228 fp=0xc0008307d8 sp=0xc0008307b8 pc=0x4185e8 2022-12-06_11:05:48.53400 reflect.mapiterinit(0x2?, 0xc000830860?, 0x467fc5?) 2022-12-06_11:05:48.53404 /usr/local/go/src/runtime/map.go:1373 +0x19 fp=0xc000830800 sp=0xc0008307d8 pc=0x46b6f9 2022-12-06_11:05:48.53407 github.com/modern-go/reflect2.(*UnsafeMapType).UnsafeIterate(...) 2022-12-06_11:05:48.53411 /root/go/pkg/mod/github.com/modern-go/reflect2@v1.0.1/unsafe_map.go:112 2022-12-06_11:05:48.53414 github.com/json-iterator/go.(*sortKeysMapEncoder).Encode(0xc000ce4420, 0xc0000be780, 0xc000dd32c0) 2022-12-06_11:05:48.53418 /root/go/pkg/mod/github.com/json-iterator/go@v1.1.10/reflect_map.go:291 +0x225 fp=0xc000830970 sp=0xc000830800 pc=0xf30385 2022-12-06_11:05:48.53422 github.com/json-iterator/go.(*structFieldEncoder).Encode(0xc000ce44e0, 0x1c60f50?, 0xc000dd32c0) 2022-12-06_11:05:48.53426 /root/go/pkg/mod/github.com/json-iterator/go@v1.1.10/reflect_struct_encoder.go:110 +0x56 fp=0xc0008309e8 sp=0xc000830970 pc=0xf3dad6 2022-12-06_11:05:48.53430 github.com/json-iterator/go.(*structEncoder).Encode(0xc000ce4570, 0x0?, 0xc000dd32c0) 2022-12-06_11:05:48.53433 /root/go/pkg/mod/github.com/json-iterator/go@v1.1.10/reflect_struct_encoder.go:158 +0x765 fp=0xc000830ad0 sp=0xc0008309e8 pc=0xf3e4e5 2022-12-06_11:05:48.53438 github.com/json-iterator/go.(*OptionalEncoder).Encode(0xc0004aa1e0?, 0x0?, 0x0?) 2022-12-06_11:05:48.53442 /root/go/pkg/mod/github.com/json-iterator/go@v1.1.10/reflect_optional.go:70 +0xa4 fp=0xc000830b20 sp=0xc000830ad0 pc=0xf356e4 2022-12-06_11:05:48.53445 github.com/json-iterator/go.(*onePtrEncoder).Encode(0xc000761460, 0xc0000be780, 0xc000ecbfb0?) 2022-12-06_11:05:48.53449 /root/go/pkg/mod/github.com/json-iterator/go@v1.1.10/reflect.go:219 +0x82 fp=0xc000830b58 sp=0xc000830b20 pc=0xf28842 2022-12-06_11:05:48.53452 github.com/json-iterator/go.(*Stream).WriteVal(0xc000dd32c0, {0x1d1a3a0, 0xc0000be780}) 2022-12-06_11:05:48.53456 /root/go/pkg/mod/github.com/json-iterator/go@v1.1.10/reflect.go:98 +0x158 fp=0xc000830bc8 sp=0xc000830b58 pc=0xf27b58 2022-12-06_11:05:48.53460 github.com/json-iterator/go.(*Encoder).Encode(0xc000a08348, {0x1d1a3a0?, 0xc0000be780?}) 2022-12-06_11:05:48.53463 /root/go/pkg/mod/github.com/json-iterator/go@v1.1.10/adapter.go:127 +0x32 fp=0xc000830c18 sp=0xc000830bc8 pc=0xf13912 2022-12-06_11:05:48.53467 github.com/grafana/grafana/pkg/api/response.StreamingResponse.WriteTo({{0x1d1a3a0?, 0xc0000be780?}, 0x1f84340?, 0xc000ecbf20?}, 0xc0003da4c0) 2022-12-06_11:05:48.53470 /var/cache/omnibus/src/grafana/pkg/api/response/response.go:111 +0x1d7 fp=0xc000830d20 sp=0xc000830c18 pc=0x104ed17 2022-12-06_11:05:48.53475 github.com/grafana/grafana/pkg/api/response.(*StreamingResponse).WriteTo(0x1f84340?, 0xc0000a0c90?) 2022-12-06_11:05:48.53479 :1 +0x45 fp=0xc000830d58 sp=0xc000830d20 pc=0x104f625 2022-12-06_11:05:48.53484 github.com/grafana/grafana/pkg/api/routing.Wrap.func1(0xc0003da4c0) 2022-12-06_11:05:48.53488 /var/cache/omnibus/src/grafana/pkg/api/routing/routing.go:25 +0x9d fp=0xc000830d90 sp=0xc000830d58 pc=0x12dd1fd 2022-12-06_11:05:48.53492 runtime.call16(0xc00002da10, 0xc0009f2318, 0x0, 0x0, 0x0, 0x8, 0xc0008312c0) 2022-12-06_11:05:48.53495 /usr/local/go/src/runtime/asm_amd64.s:701 +0x49 fp=0xc000830db0 sp=0xc000830d90 pc=0x46ffc9 2022-12-06_11:05:48.53499 runtime.reflectcall(0x2270820?, 0xc0003da4c0?, 0x1?, 0x22b48bc?, 0x0?, 0x12?, 0x2270820?) 2022-12-06_11:05:48.53502 :1 +0x3c fp=0xc000830df0 sp=0xc000830db0 pc=0x47453c 2022-12-06_11:05:48.53506 reflect.Value.call({0x1d7f740?, 0xc0009f2318?, 0x7fceaf01da68?}, {0x2291c88, 0x4}, {0xc00000f7b8, 0x1, 0x0?}) 2022-12-06_11:05:48.53509 /usr/local/go/src/reflect/value.go:556 +0x845 fp=0xc0008313e0 sp=0xc000830df0 pc=0x49b3c5 2022-12-06_11:05:48.53513 reflect.Value.Call({0x1d7f740?, 0xc0009f2318?, 0x2270820?}, {0xc00000f7b8, 0x1, 0x1}) 2022-12-06_11:05:48.53518 /usr/local/go/src/reflect/value.go:339 +0xbf fp=0xc000831458 sp=0xc0008313e0 pc=0x49a93f 2022-12-06_11:05:48.53553 github.com/go-macaron/inject.(*injector).callInvoke(0x222c7d22223a2264?, {0x1d7f740?, 0xc0009f2318?}, {0x26823f8?, 0x1d7f740?}, 0x1?) 2022-12-06_11:05:48.53557 /root/go/pkg/mod/github.com/go-macaron/inject@v0.0.0-20160627170012-d8a0b8677191/inject.go:177 +0x127 fp=0xc0008314e8 sp=0xc000831458 pc=0xe158c7 2022-12-06_11:05:48.53561 github.com/go-macaron/inject.(*injector).Invoke(0x6574617473222c31?, {0x1d7f740?, 0xc0009f2318}) 2022-12-06_11:05:48.53564 /root/go/pkg/mod/github.com/go-macaron/inject@v0.0.0-20160627170012-d8a0b8677191/inject.go:137 +0xb5 fp=0xc000831568 sp=0xc0008314e8 pc=0xe15515 2022-12-06_11:05:48.53568 gopkg.in/macaron%2ev1.(*Context).run(0xc0002566c0) 2022-12-06_11:05:48.53572 /var/cache/omnibus/src/grafana/pkg/macaron/context.go:125 +0x73 fp=0xc0008315c0 sp=0xc000831568 pc=0xe17533 2022-12-06_11:05:48.53576 gopkg.in/macaron%2ev1.(*Context).Next(...) 2022-12-06_11:05:48.53580 /var/cache/omnibus/src/grafana/pkg/macaron/context.go:115 2022-12-06_11:05:48.53584 github.com/grafana/grafana/pkg/middleware.RequestMetrics.func1.1({0x7fce84ece958?, 0xc0000a6d20}, 0xc0009d6600, 0xc0002566c0) 2022-12-06_11:05:48.53589 /var/cache/omnibus/src/grafana/pkg/middleware/request_metrics.go:54 +0x134 fp=0xc000831ca0 sp=0xc0008315c0 pc=0x1330054 2022-12-06_11:05:48.53593 runtime.call32(0xc000bff770, 0xc0009eeb40, 0x0, 0x0, 0x0, 0x20, 0xc0008321e0) 2022-12-06_11:05:48.53596 /usr/local/go/src/runtime/asm_amd64.s:702 +0x49 fp=0xc000831cd0 sp=0xc000831ca0 pc=0x470069 2022-12-06_11:05:48.53600 runtime.reflectcall(0x226f5e0?, 0xc0002566c0?, 0x4?, 0x22b48bc?, 0x0?, 0x12?, 0x226f5e0?) 2022-12-06_11:05:48.53604 :1 +0x3c fp=0xc000831d10 sp=0xc000831cd0 pc=0x47453c 2022-12-06_11:05:48.53608 reflect.Value.call({0x1e7fde0?, 0xc0009eeb40?, 0x7fceaf01da68?}, {0x2291c88, 0x4}, {0xc0000a7040, 0x3, 0x2?}) 2022-12-06_11:05:48.53612 /usr/local/go/src/reflect/value.go:556 +0x845 fp=0xc000832300 sp=0xc000831d10 pc=0x49b3c5 2022-12-06_11:05:48.53615 reflect.Value.Call({0x1e7fde0?, 0xc0009eeb40?, 0x226f5e0?}, {0xc0000a7040, 0x3, 0x3}) 2022-12-06_11:05:48.53619 /usr/local/go/src/reflect/value.go:339 +0xbf fp=0xc000832378 sp=0xc000832300 pc=0x49a93f 2022-12-06_11:05:48.53623 github.com/go-macaron/inject.(*injector).callInvoke(0xc000060c00?, {0x1e7fde0?, 0xc0009eeb40?}, {0x26823f8?, 0x1e7fde0?}, 0x3?) 2022-12-06_11:05:48.53627 /root/go/pkg/mod/github.com/go-macaron/inject@v0.0.0-20160627170012-d8a0b8677191/inject.go:177 +0x127 fp=0xc000832408 sp=0xc000832378 pc=0xe158c7 2022-12-06_11:05:48.53632 github.com/go-macaron/inject.(*injector).Invoke(0xc000060c00?, {0x1e7fde0?, 0xc0009eeb40}) 2022-12-06_11:05:48.53636 /root/go/pkg/mod/github.com/go-macaron/inject@v0.0.0-20160627170012-d8a0b8677191/inject.go:137 +0xb5 fp=0xc000832488 sp=0xc000832408 pc=0xe15515 2022-12-06_11:05:48.53640 gopkg.in/macaron%2ev1.(*Context).run(0xc0002566c0) 2022-12-06_11:05:48.53643 /var/cache/omnibus/src/grafana/pkg/macaron/context.go:125 +0x73 fp=0xc0008324e0 sp=0xc000832488 pc=0xe17533 2022-12-06_11:05:48.53650 gopkg.in/macaron%2ev1.(*Context).Next(...) 2022-12-06_11:05:48.53654 /var/cache/omnibus/src/grafana/pkg/macaron/context.go:115 2022-12-06_11:05:48.53657 github.com/grafana/grafana/pkg/middleware.RequestTracing.func1({0x7fce84ece958?, 0xc0000a6d20}, 0xc0009d6600, 0xc0002566c0) 2022-12-06_11:05:48.53661 /var/cache/omnibus/src/grafana/pkg/middleware/request_tracing.go:25 +0x4c5 fp=0xc000832628 sp=0xc0008324e0 pc=0x1331a45 2022-12-06_11:05:48.53665 runtime.call32(0xc000bff770, 0xc0009f2330, 0x0, 0x0, 0x0, 0x20, 0xc000832b68) 2022-12-06_11:05:48.53669 /usr/local/go/src/runtime/asm_amd64.s:702 +0x49 fp=0xc000832658 sp=0xc000832628 pc=0x470069 2022-12-06_11:05:48.53673 runtime.reflectcall(0x226f5e0?, 0xc0002566c0?, 0x4?, 0x22b48bc?, 0x0?, 0x12?, 0x226f5e0?) 2022-12-06_11:05:48.53678 :1 +0x3c fp=0xc000832698 sp=0xc000832658 pc=0x47453c 2022-12-06_11:05:48.53682 reflect.Value.call({0x1e7fde0?, 0xc0009f2330?, 0x7fceaf01da68?}, {0x2291c88, 0x4}, {0xc0000a6ff0, 0x3, 0xc000d487a0?}) 2022-12-06_11:05:48.53686 /usr/local/go/src/reflect/value.go:556 +0x845 fp=0xc000832c88 sp=0xc000832698 pc=0x49b3c5 2022-12-06_11:05:48.53689 reflect.Value.Call({0x1e7fde0?, 0xc0009f2330?, 0x226f5e0?}, {0xc0000a6ff0, 0x3, 0x3}) 2022-12-06_11:05:48.53693 /usr/local/go/src/reflect/value.go:339 +0xbf fp=0xc000832d00 sp=0xc000832c88 pc=0x49a93f 2022-12-06_11:05:48.53697 github.com/go-macaron/inject.(*injector).callInvoke(0xc000ef2db8?, {0x1e7fde0?, 0xc0009f2330?}, {0x26823f8?, 0x1e7fde0?}, 0x3?) 2022-12-06_11:05:48.53701 /root/go/pkg/mod/github.com/go-macaron/inject@v0.0.0-20160627170012-d8a0b8677191/inject.go:177 +0x127 fp=0xc000832d90 sp=0xc000832d00 pc=0xe158c7 2022-12-06_11:05:48.53705 github.com/go-macaron/inject.(*injector).Invoke(0xc000d6a6c0?, {0x1e7fde0?, 0xc0009f2330}) 2022-12-06_11:05:48.53709 /root/go/pkg/mod/github.com/go-macaron/inject@v0.0.0-20160627170012-d8a0b8677191/inject.go:137 +0xb5 fp=0xc000832e10 sp=0xc000832d90 pc=0xe15515 2022-12-06_11:05:48.53714 gopkg.in/macaron%2ev1.(*Context).run(0xc0002566c0) 2022-12-06_11:05:48.53717 /var/cache/omnibus/src/grafana/pkg/macaron/context.go:125 +0x73 fp=0xc000832e68 sp=0xc000832e10 pc=0xe17533 2022-12-06_11:05:48.53721 gopkg.in/macaron%2ev1.(*Context).Next(...) 2022-12-06_11:05:48.53725 /var/cache/omnibus/src/grafana/pkg/macaron/context.go:115 2022-12-06_11:05:48.53729 github.com/grafana/grafana/pkg/middleware.Recovery.func1(0x10?) 2022-12-06_11:05:48.53733 /var/cache/omnibus/src/grafana/pkg/middleware/recovery.go:166 +0x6b fp=0xc000832ea8 sp=0xc000832e68 pc=0x132f1eb 2022-12-06_11:05:48.53737 gopkg.in/macaron%2ev1.ContextInvoker.Invoke(0x226f5e0?, {0xc000d48150?, 0x226f5e0?, 0x203000?}) 2022-12-06_11:05:48.53740 /var/cache/omnibus/src/grafana/pkg/macaron/context.go:81 +0x3c fp=0xc000832ed0 sp=0xc000832ea8 pc=0xe173dc 2022-12-06_11:05:48.53744 github.com/go-macaron/inject.(*injector).fastInvoke(0x120?, {0x7fce84ece988?, 0xc000210020?}, {0x26823f8?, 0x1efd560?}, 0x1?) 2022-12-06_11:05:48.53748 /root/go/pkg/mod/github.com/go-macaron/inject@v0.0.0-20160627170012-d8a0b8677191/inject.go:157 +0x95 fp=0xc000832f40 sp=0xc000832ed0 pc=0xe15635 2022-12-06_11:05:48.53752 github.com/go-macaron/inject.(*injector).Invoke(0x1?, {0x1efd560?, 0xc000210020}) 2022-12-06_11:05:48.53756 /root/go/pkg/mod/github.com/go-macaron/inject@v0.0.0-20160627170012-d8a0b8677191/inject.go:135 +0xfd fp=0xc000832fc0 sp=0xc000832f40 pc=0xe1555d 2022-12-06_11:05:48.53761 gopkg.in/macaron%2ev1.(*Context).run(0xc0002566c0) 2022-12-06_11:05:48.53765 /var/cache/omnibus/src/grafana/pkg/macaron/context.go:125 +0x73 fp=0xc000833018 sp=0xc000832fc0 pc=0xe17533 2022-12-06_11:05:48.53769 gopkg.in/macaron%2ev1.(*Context).Next(...) 2022-12-06_11:05:48.53773 /var/cache/omnibus/src/grafana/pkg/macaron/context.go:115 2022-12-06_11:05:48.53777 github.com/grafana/grafana/pkg/middleware.Logger.func1({0x7fce84ece958?, 0xc0000a6d20}, 0xc0009d6600, 0xc0002566c0) 2022-12-06_11:05:48.53782 /var/cache/omnibus/src/grafana/pkg/middleware/logger.go:35 +0x12f fp=0xc000833108 sp=0xc000833018 pc=0x132d08f 2022-12-06_11:05:48.53789 runtime.call32(0xc000bff770, 0xc000210000, 0x0, 0x0, 0x0, 0x20, 0xc000833648) 2022-12-06_11:05:48.53793 /usr/local/go/src/runtime/asm_amd64.s:702 +0x49 fp=0xc000833138 sp=0xc000833108 pc=0x470069 2022-12-06_11:05:48.53797 runtime.reflectcall(0x226f5e0?, 0xc0002566c0?, 0x4?, 0x22b48bc?, 0x0?, 0x12?, 0x226f5e0?) 2022-12-06_11:05:48.53801 :1 +0x3c fp=0xc000833178 sp=0xc000833138 pc=0x47453c 2022-12-06_11:05:48.53805 reflect.Value.call({0x1e7fde0?, 0xc000210000?, 0x13?}, {0x2291c88, 0x4}, {0xc0000a6d70, 0x3, 0x3?}) 2022-12-06_11:05:48.53810 /usr/local/go/src/reflect/value.go:556 +0x845 fp=0xc000833768 sp=0xc000833178 pc=0x49b3c5 2022-12-06_11:05:48.53814 reflect.Value.Call({0x1e7fde0?, 0xc000210000?, 0x226f5e0?}, {0xc0000a6d70, 0x3, 0x3}) 2022-12-06_11:05:48.53818 /usr/local/go/src/reflect/value.go:339 +0xbf fp=0xc0008337e0 sp=0xc000833768 pc=0x49a93f 2022-12-06_11:05:48.53821 github.com/go-macaron/inject.(*injector).callInvoke(0x16?, {0x1e7fde0?, 0xc000210000?}, {0x26823f8?, 0x1e7fde0?}, 0x3?) 2022-12-06_11:05:48.53825 /root/go/pkg/mod/github.com/go-macaron/inject@v0.0.0-20160627170012-d8a0b8677191/inject.go:177 +0x127 fp=0xc000833870 sp=0xc0008337e0 pc=0xe158c7 2022-12-06_11:05:48.53829 github.com/go-macaron/inject.(*injector).Invoke(0xc000060c00?, {0x1e7fde0?, 0xc000210000}) 2022-12-06_11:05:48.53833 /root/go/pkg/mod/github.com/go-macaron/inject@v0.0.0-20160627170012-d8a0b8677191/inject.go:137 +0xb5 fp=0xc0008338f0 sp=0xc000833870 pc=0xe15515 2022-12-06_11:05:48.53837 gopkg.in/macaron%2ev1.(*Context).run(0xc0002566c0) 2022-12-06_11:05:48.53841 /var/cache/omnibus/src/grafana/pkg/macaron/context.go:125 +0x73 fp=0xc000833948 sp=0xc0008338f0 pc=0xe17533 2022-12-06_11:05:48.53845 gopkg.in/macaron%2ev1.(*Router).Handle.func1({0x266f7e8?, 0xc0002ac700?}, 0x2290cd5?, 0x0) 2022-12-06_11:05:48.53850 /var/cache/omnibus/src/grafana/pkg/macaron/router.go:187 +0x285 fp=0xc0008339d8 sp=0xc000833948 pc=0xe22305 2022-12-06_11:05:48.53854 gopkg.in/macaron%2ev1.(*Router).ServeHTTP(0xc00095e7e0, {0x266f7e8, 0xc0002ac700}, 0xc0009d6600) 2022-12-06_11:05:48.53858 /var/cache/omnibus/src/grafana/pkg/macaron/router.go:295 +0x1b6 fp=0xc000833a38 sp=0xc0008339d8 pc=0xe234b6 2022-12-06_11:05:48.53862 gopkg.in/macaron%2ev1.(*Macaron).ServeHTTP(0xc0008df080, {0x266f7e8, 0xc0002ac700}, 0xc0009d6600) 2022-12-06_11:05:48.53866 /var/cache/omnibus/src/grafana/pkg/macaron/macaron.go:220 +0x165 fp=0xc000833a98 sp=0xc000833a38 pc=0xe1bfe5 2022-12-06_11:05:48.53870 net/http.serverHandler.ServeHTTP({0x2669138?}, {0x266f7e8, 0xc0002ac700}, 0xc0009d6600) 2022-12-06_11:05:48.53873 /usr/local/go/src/net/http/server.go:2916 +0x43b fp=0xc000833b58 sp=0xc000833a98 pc=0x6f6efb 2022-12-06_11:05:48.53877 net/http.(*conn).serve(0xc0004aae60, {0x2670ba8, 0xc00053fda0}) 2022-12-06_11:05:48.53881 /usr/local/go/src/net/http/server.go:1966 +0x5d7 fp=0xc000833fb8 sp=0xc000833b58 pc=0x6f23b7 2022-12-06_11:05:48.53885 net/http.(*Server).Serve.func3() 2022-12-06_11:05:48.53889 /usr/local/go/src/net/http/server.go:3071 +0x2e fp=0xc000833fe0 sp=0xc000833fb8 pc=0x6f784e 2022-12-06_11:05:48.53893 runtime.goexit() 2022-12-06_11:05:48.53898 /usr/local/go/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc000833fe8 sp=0xc000833fe0 pc=0x471ca1 2022-12-06_11:05:48.53902 created by net/http.(*Server).Serve 2022-12-06_11:05:48.53906 /usr/local/go/src/net/http/server.go:3071 +0x4db 2022-12-06_11:05:48.53909 2022-12-06_11:05:48.53913 goroutine 1 [semacquire, 20 minutes]: 2022-12-06_11:05:48.53917 sync.runtime_Semacquire(0x22c0073?) 2022-12-06_11:05:48.53922 /usr/local/go/src/runtime/sema.go:56 +0x25 2022-12-06_11:05:48.53926 sync.(*WaitGroup).Wait(0x0?) 2022-12-06_11:05:48.53930 /usr/local/go/src/sync/waitgroup.go:136 +0x52 2022-12-06_11:05:48.53934 golang.org/x/sync/errgroup.(*Group).Wait(0xc000339980) 2022-12-06_11:05:48.53938 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:40 +0x27 2022-12-06_11:05:48.53942 github.com/grafana/grafana/pkg/server.(*Server).Run.func2() 2022-12-06_11:05:48.53947 /var/cache/omnibus/src/grafana/pkg/server/server.go:198 +0x5a 2022-12-06_11:05:48.53951 github.com/grafana/grafana/pkg/server.(*Server).Run(0xc000292750) 2022-12-06_11:05:48.53955 /var/cache/omnibus/src/grafana/pkg/server/server.go:208 +0x21d 2022-12-06_11:05:48.53959 main.executeServer({0x7ffff44e3eca, 0x23}, {0x0, 0x0}, {0x0, 0x0}, {0x2298ba5, 0x7}, 0x226e540?) 2022-12-06_11:05:48.53963 /var/cache/omnibus/src/grafana/pkg/cmd/grafana-server/main.go:161 +0x6e9 2022-12-06_11:05:48.53967 main.main() 2022-12-06_11:05:48.53971 /var/cache/omnibus/src/grafana/pkg/cmd/grafana-server/main.go:98 +0x5c5 2022-12-06_11:05:48.53976 2022-12-06_11:05:48.53980 goroutine 122 [select]: 2022-12-06_11:05:48.53983 go.opencensus.io/stats/view.(*worker).start(0xc0003ca300) 2022-12-06_11:05:48.53987 /root/go/pkg/mod/go.opencensus.io@v0.22.6/stats/view/worker.go:276 +0xad 2022-12-06_11:05:48.53991 created by go.opencensus.io/stats/view.init.0 2022-12-06_11:05:48.53995 /root/go/pkg/mod/go.opencensus.io@v0.22.6/stats/view/worker.go:34 +0x8d 2022-12-06_11:05:48.53999 2022-12-06_11:05:48.54002 goroutine 125 [select, 2 minutes]: 2022-12-06_11:05:48.54006 github.com/patrickmn/go-cache.(*janitor).Run(0xc0008c8130, 0xc00003f2c0?) 2022-12-06_11:05:48.54010 /root/go/pkg/mod/github.com/patrickmn/go-cache@v2.1.0+incompatible/cache.go:1079 +0x85 2022-12-06_11:05:48.54014 created by github.com/patrickmn/go-cache.runJanitor 2022-12-06_11:05:48.54018 /root/go/pkg/mod/github.com/patrickmn/go-cache@v2.1.0+incompatible/cache.go:1099 +0xed 2022-12-06_11:05:48.54022 2022-12-06_11:05:48.54026 goroutine 126 [select, 20 minutes]: 2022-12-06_11:05:48.54029 database/sql.(*DB).connectionOpener(0xc0008d0820, {0x2670b00, 0xc0008be1c0}) 2022-12-06_11:05:48.54033 /usr/local/go/src/database/sql/sql.go:1226 +0x8d 2022-12-06_11:05:48.54037 created by database/sql.OpenDB 2022-12-06_11:05:48.54041 /usr/local/go/src/database/sql/sql.go:794 +0x18d 2022-12-06_11:05:48.54045 2022-12-06_11:05:48.54049 goroutine 127 [select, 20 minutes]: 2022-12-06_11:05:48.54053 database/sql.(*DB).connectionCleaner(0xc0008d0820, 0x0?) 2022-12-06_11:05:48.54056 /usr/local/go/src/database/sql/sql.go:1069 +0xa7 2022-12-06_11:05:48.54061 created by database/sql.(*DB).startCleanerLocked 2022-12-06_11:05:48.54065 /usr/local/go/src/database/sql/sql.go:1056 +0xff 2022-12-06_11:05:48.54068 2022-12-06_11:05:48.54072 goroutine 146 [select, 20 minutes]: 2022-12-06_11:05:48.54076 github.com/patrickmn/go-cache.(*janitor).Run(0xc0009e3a20, 0x0?) 2022-12-06_11:05:48.54079 /root/go/pkg/mod/github.com/patrickmn/go-cache@v2.1.0+incompatible/cache.go:1079 +0x85 2022-12-06_11:05:48.54083 created by github.com/patrickmn/go-cache.runJanitor 2022-12-06_11:05:48.54087 /root/go/pkg/mod/github.com/patrickmn/go-cache@v2.1.0+incompatible/cache.go:1099 +0xed 2022-12-06_11:05:48.54091 2022-12-06_11:05:48.54095 goroutine 147 [select]: 2022-12-06_11:05:48.54099 github.com/grafana/grafana/pkg/services/alerting.(*Ticker).run(0xc000bb6410) 2022-12-06_11:05:48.54102 /var/cache/omnibus/src/grafana/pkg/services/alerting/ticker.go:55 +0x1a7 2022-12-06_11:05:48.54106 created by github.com/grafana/grafana/pkg/services/alerting.NewTicker 2022-12-06_11:05:48.54110 /var/cache/omnibus/src/grafana/pkg/services/alerting/ticker.go:39 +0x185 2022-12-06_11:05:48.54114 2022-12-06_11:05:48.54118 goroutine 136 [select, 20 minutes]: 2022-12-06_11:05:48.54122 github.com/grafana/grafana/pkg/services/alerting.(*AlertEngine).runJobDispatcher(0xc0002565a0, {0x2670b00, 0xc0000a3040}) 2022-12-06_11:05:48.54126 /var/cache/omnibus/src/grafana/pkg/services/alerting/engine.go:99 +0x105 2022-12-06_11:05:48.54130 github.com/grafana/grafana/pkg/services/alerting.(*AlertEngine).Run.func2() 2022-12-06_11:05:48.54142 /var/cache/omnibus/src/grafana/pkg/services/alerting/engine.go:64 +0x25 2022-12-06_11:05:48.54147 golang.org/x/sync/errgroup.(*Group).Go.func1() 2022-12-06_11:05:48.54151 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67 2022-12-06_11:05:48.54157 created by golang.org/x/sync/errgroup.(*Group).Go 2022-12-06_11:05:48.54161 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d 2022-12-06_11:05:48.54165 2022-12-06_11:05:48.54169 goroutine 135 [select]: 2022-12-06_11:05:48.54173 github.com/grafana/grafana/pkg/services/alerting.(*AlertEngine).alertingTicker(0xc0002565a0, {0x2670b00, 0xc0000a3040}) 2022-12-06_11:05:48.54177 /var/cache/omnibus/src/grafana/pkg/services/alerting/engine.go:80 +0x145 2022-12-06_11:05:48.54182 github.com/grafana/grafana/pkg/services/alerting.(*AlertEngine).Run.func1() 2022-12-06_11:05:48.54186 /var/cache/omnibus/src/grafana/pkg/services/alerting/engine.go:63 +0x25 2022-12-06_11:05:48.54190 golang.org/x/sync/errgroup.(*Group).Go.func1() 2022-12-06_11:05:48.54194 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67 2022-12-06_11:05:48.54198 created by golang.org/x/sync/errgroup.(*Group).Go 2022-12-06_11:05:48.54202 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d 2022-12-06_11:05:48.54207 2022-12-06_11:05:48.54211 goroutine 134 [syscall, 20 minutes]: 2022-12-06_11:05:48.54215 os/signal.signal_recv() 2022-12-06_11:05:48.54218 /usr/local/go/src/runtime/sigqueue.go:151 +0x2f 2022-12-06_11:05:48.54222 os/signal.loop() 2022-12-06_11:05:48.54226 /usr/local/go/src/os/signal/signal_unix.go:23 +0x19 2022-12-06_11:05:48.54230 created by os/signal.Notify.func1.1 2022-12-06_11:05:48.54234 /usr/local/go/src/os/signal/signal.go:151 +0x2a 2022-12-06_11:05:48.54238 2022-12-06_11:05:48.54241 goroutine 154 [select, 20 minutes]: 2022-12-06_11:05:48.54245 main.listenToSystemSignals(0x0?) 2022-12-06_11:05:48.54249 /var/cache/omnibus/src/grafana/pkg/cmd/grafana-server/main.go:190 +0x155 2022-12-06_11:05:48.54253 created by main.executeServer 2022-12-06_11:05:48.54257 /var/cache/omnibus/src/grafana/pkg/cmd/grafana-server/main.go:159 +0x6dc 2022-12-06_11:05:48.54261 2022-12-06_11:05:48.54265 goroutine 155 [IO wait]: 2022-12-06_11:05:48.54268 internal/poll.runtime_pollWait(0x7fce863f81d8, 0x72) 2022-12-06_11:05:48.54272 /usr/local/go/src/runtime/netpoll.go:302 +0x89 2022-12-06_11:05:48.54276 internal/poll.(*pollDesc).wait(0xc00056a380?, 0xc00004ea00?, 0x0) 2022-12-06_11:05:48.54280 /usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x32 2022-12-06_11:05:48.54283 internal/poll.(*pollDesc).waitRead(...) 2022-12-06_11:05:48.54287 /usr/local/go/src/internal/poll/fd_poll_runtime.go:88 2022-12-06_11:05:48.54291 internal/poll.(*FD).Accept(0xc00056a380) 2022-12-06_11:05:48.54295 /usr/local/go/src/internal/poll/fd_unix.go:614 +0x22c 2022-12-06_11:05:48.54298 net.(*netFD).accept(0xc00056a380) 2022-12-06_11:05:48.54304 /usr/local/go/src/net/fd_unix.go:172 +0x35 2022-12-06_11:05:48.54308 net.(*TCPListener).accept(0xc000bd6240) 2022-12-06_11:05:48.54312 /usr/local/go/src/net/tcpsock_posix.go:139 +0x28 2022-12-06_11:05:48.54315 net.(*TCPListener).Accept(0xc000bd6240) 2022-12-06_11:05:48.54320 /usr/local/go/src/net/tcpsock.go:288 +0x3d 2022-12-06_11:05:48.54325 net/http.(*Server).Serve(0xc0008d60e0, {0x266f5a8, 0xc000bd6240}) 2022-12-06_11:05:48.54329 /usr/local/go/src/net/http/server.go:3039 +0x385 2022-12-06_11:05:48.54333 github.com/grafana/grafana/pkg/api.(*HTTPServer).Run(0xc000107b80, {0x2670b00?, 0xc000339940}) 2022-12-06_11:05:48.54337 /var/cache/omnibus/src/grafana/pkg/api/http_server.go:149 +0x72b 2022-12-06_11:05:48.54340 github.com/grafana/grafana/pkg/server.(*Server).Run.func1() 2022-12-06_11:05:48.54344 /var/cache/omnibus/src/grafana/pkg/server/server.go:177 +0x49 2022-12-06_11:05:48.54348 golang.org/x/sync/errgroup.(*Group).Go.func1() 2022-12-06_11:05:48.54352 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67 2022-12-06_11:05:48.54356 created by golang.org/x/sync/errgroup.(*Group).Go 2022-12-06_11:05:48.54359 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d 2022-12-06_11:05:48.54365 2022-12-06_11:05:48.54368 goroutine 156 [chan receive, 20 minutes]: 2022-12-06_11:05:48.54371 github.com/grafana/grafana/pkg/services/rendering.(*RenderingService).Run(0xc00020f180, {0x2670b00, 0xc000339940}) 2022-12-06_11:05:48.54376 /var/cache/omnibus/src/grafana/pkg/services/rendering/rendering.go:104 +0x1b9 2022-12-06_11:05:48.54379 github.com/grafana/grafana/pkg/server.(*Server).Run.func1() 2022-12-06_11:05:48.54382 /var/cache/omnibus/src/grafana/pkg/server/server.go:177 +0x49 2022-12-06_11:05:48.54385 golang.org/x/sync/errgroup.(*Group).Go.func1() 2022-12-06_11:05:48.54388 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67 2022-12-06_11:05:48.54391 created by golang.org/x/sync/errgroup.(*Group).Go 2022-12-06_11:05:48.54394 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d 2022-12-06_11:05:48.54398 2022-12-06_11:05:48.54401 goroutine 157 [chan receive, 20 minutes]: 2022-12-06_11:05:48.54405 github.com/grafana/grafana/pkg/plugins/backendplugin.(*manager).Run(0x0?, {0x2670b00, 0xc000339940}) 2022-12-06_11:05:48.54409 /var/cache/omnibus/src/grafana/pkg/plugins/backendplugin/manager.go:75 +0x3f 2022-12-06_11:05:48.54413 github.com/grafana/grafana/pkg/server.(*Server).Run.func1() 2022-12-06_11:05:48.54417 /var/cache/omnibus/src/grafana/pkg/server/server.go:177 +0x49 2022-12-06_11:05:48.54421 golang.org/x/sync/errgroup.(*Group).Go.func1() 2022-12-06_11:05:48.54425 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67 2022-12-06_11:05:48.54430 created by golang.org/x/sync/errgroup.(*Group).Go 2022-12-06_11:05:48.54434 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d 2022-12-06_11:05:48.54438 2022-12-06_11:05:48.54441 goroutine 158 [select, 2 minutes]: 2022-12-06_11:05:48.54444 github.com/grafana/grafana/pkg/infra/remotecache.(*databaseCache).Run(0x0?, {0x2670b00, 0xc000339940}) 2022-12-06_11:05:48.54448 /var/cache/omnibus/src/grafana/pkg/infra/remotecache/database_storage.go:32 +0x9f 2022-12-06_11:05:48.54451 github.com/grafana/grafana/pkg/infra/remotecache.(*RemoteCache).Run(0x0?, {0x2670b00, 0xc000339940}) 2022-12-06_11:05:48.54455 /var/cache/omnibus/src/grafana/pkg/infra/remotecache/remotecache.go:94 +0x58 2022-12-06_11:05:48.54459 github.com/grafana/grafana/pkg/server.(*Server).Run.func1() 2022-12-06_11:05:48.54463 /var/cache/omnibus/src/grafana/pkg/server/server.go:177 +0x49 2022-12-06_11:05:48.54467 golang.org/x/sync/errgroup.(*Group).Go.func1() 2022-12-06_11:05:48.54471 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67 2022-12-06_11:05:48.54475 created by golang.org/x/sync/errgroup.(*Group).Go 2022-12-06_11:05:48.54479 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d 2022-12-06_11:05:48.54482 2022-12-06_11:05:48.54486 goroutine 159 [select, 2 minutes]: 2022-12-06_11:05:48.54491 github.com/grafana/grafana/pkg/plugins.(*PluginManager).Run(0x35700000007?, {0x2670b00, 0xc000339940}) 2022-12-06_11:05:48.54495 /var/cache/omnibus/src/grafana/pkg/plugins/plugins.go:164 +0xc5 2022-12-06_11:05:48.54499 github.com/grafana/grafana/pkg/server.(*Server).Run.func1() 2022-12-06_11:05:48.54503 /var/cache/omnibus/src/grafana/pkg/server/server.go:177 +0x49 2022-12-06_11:05:48.54507 golang.org/x/sync/errgroup.(*Group).Go.func1() 2022-12-06_11:05:48.54511 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67 2022-12-06_11:05:48.54515 created by golang.org/x/sync/errgroup.(*Group).Go 2022-12-06_11:05:48.54519 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d 2022-12-06_11:05:48.54523 2022-12-06_11:05:48.54526 goroutine 160 [semacquire, 20 minutes]: 2022-12-06_11:05:48.54530 sync.runtime_Semacquire(0xc00044fa00?) 2022-12-06_11:05:48.54534 /usr/local/go/src/runtime/sema.go:56 +0x25 2022-12-06_11:05:48.54538 sync.(*WaitGroup).Wait(0xc0000a3001?) 2022-12-06_11:05:48.54543 /usr/local/go/src/sync/waitgroup.go:136 +0x52 2022-12-06_11:05:48.54547 golang.org/x/sync/errgroup.(*Group).Wait(0xc0000a3100) 2022-12-06_11:05:48.54550 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:40 +0x27 2022-12-06_11:05:48.54554 github.com/grafana/grafana/pkg/services/alerting.(*AlertEngine).Run(0xc0002565a0, {0x2670b00?, 0xc000339940?}) 2022-12-06_11:05:48.54559 /var/cache/omnibus/src/grafana/pkg/services/alerting/engine.go:66 +0x15f 2022-12-06_11:05:48.54563 github.com/grafana/grafana/pkg/server.(*Server).Run.func1() 2022-12-06_11:05:48.54566 /var/cache/omnibus/src/grafana/pkg/server/server.go:177 +0x49 2022-12-06_11:05:48.54570 golang.org/x/sync/errgroup.(*Group).Go.func1() 2022-12-06_11:05:48.54574 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67 2022-12-06_11:05:48.54578 created by golang.org/x/sync/errgroup.(*Group).Go 2022-12-06_11:05:48.54584 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d 2022-12-06_11:05:48.54588 2022-12-06_11:05:48.54592 goroutine 161 [select, 20 minutes]: 2022-12-06_11:05:48.54596 github.com/grafana/grafana/pkg/services/auth.(*UserAuthTokenService).Run(0xc000522d80, {0x2670b00, 0xc000339940}) 2022-12-06_11:05:48.54600 /var/cache/omnibus/src/grafana/pkg/services/auth/token_cleanup.go:25 +0x205 2022-12-06_11:05:48.54604 github.com/grafana/grafana/pkg/server.(*Server).Run.func1() 2022-12-06_11:05:48.54608 /var/cache/omnibus/src/grafana/pkg/server/server.go:177 +0x49 2022-12-06_11:05:48.54611 golang.org/x/sync/errgroup.(*Group).Go.func1() 2022-12-06_11:05:48.54615 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67 2022-12-06_11:05:48.54621 created by golang.org/x/sync/errgroup.(*Group).Go 2022-12-06_11:05:48.54625 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d 2022-12-06_11:05:48.54628 2022-12-06_11:05:48.54632 goroutine 162 [chan receive, 20 minutes]: 2022-12-06_11:05:48.54636 github.com/grafana/grafana/pkg/infra/metrics.(*InternalMetricsService).Run(0x0?, {0x2670b00, 0xc000339940}) 2022-12-06_11:05:48.54640 /var/cache/omnibus/src/grafana/pkg/infra/metrics/service.go:52 +0x170 2022-12-06_11:05:48.54644 github.com/grafana/grafana/pkg/server.(*Server).Run.func1() 2022-12-06_11:05:48.54648 /var/cache/omnibus/src/grafana/pkg/server/server.go:177 +0x49 2022-12-06_11:05:48.54652 golang.org/x/sync/errgroup.(*Group).Go.func1() 2022-12-06_11:05:48.54655 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67 2022-12-06_11:05:48.54659 created by golang.org/x/sync/errgroup.(*Group).Go 2022-12-06_11:05:48.54663 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d 2022-12-06_11:05:48.54667 2022-12-06_11:05:48.54671 goroutine 163 [select, 20 minutes]: 2022-12-06_11:05:48.54675 github.com/grafana/grafana/pkg/services/notifications.(*NotificationService).Run(0xc000249e00, {0x2670b00, 0xc000339940}) 2022-12-06_11:05:48.54679 /var/cache/omnibus/src/grafana/pkg/services/notifications/notifications.go:78 +0x118 2022-12-06_11:05:48.54684 github.com/grafana/grafana/pkg/server.(*Server).Run.func1() 2022-12-06_11:05:48.54688 /var/cache/omnibus/src/grafana/pkg/server/server.go:177 +0x49 2022-12-06_11:05:48.54691 golang.org/x/sync/errgroup.(*Group).Go.func1() 2022-12-06_11:05:48.54695 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67 2022-12-06_11:05:48.54699 created by golang.org/x/sync/errgroup.(*Group).Go 2022-12-06_11:05:48.54703 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d 2022-12-06_11:05:48.54707 2022-12-06_11:05:48.54711 goroutine 164 [chan receive, 20 minutes]: 2022-12-06_11:05:48.54715 github.com/grafana/grafana/pkg/infra/tracing.(*TracingService).Run(0xc000204a80, {0x2670b00?, 0xc000339940?}) 2022-12-06_11:05:48.54721 /var/cache/omnibus/src/grafana/pkg/infra/tracing/tracing.go:129 +0x37 2022-12-06_11:05:48.54726 github.com/grafana/grafana/pkg/server.(*Server).Run.func1() 2022-12-06_11:05:48.54730 /var/cache/omnibus/src/grafana/pkg/server/server.go:177 +0x49 2022-12-06_11:05:48.54733 golang.org/x/sync/errgroup.(*Group).Go.func1() 2022-12-06_11:05:48.54737 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67 2022-12-06_11:05:48.54742 created by golang.org/x/sync/errgroup.(*Group).Go 2022-12-06_11:05:48.54748 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d 2022-12-06_11:05:48.54752 2022-12-06_11:05:48.54756 goroutine 165 [select, 2 minutes]: 2022-12-06_11:05:48.54760 github.com/grafana/grafana/pkg/services/cleanup.(*CleanUpService).Run(0xc0004d82d0, {0x2670b00, 0xc000339940}) 2022-12-06_11:05:48.54764 /var/cache/omnibus/src/grafana/pkg/services/cleanup/cleanup.go:42 +0xca 2022-12-06_11:05:48.54767 github.com/grafana/grafana/pkg/server.(*Server).Run.func1() 2022-12-06_11:05:48.54771 /var/cache/omnibus/src/grafana/pkg/server/server.go:177 +0x49 2022-12-06_11:05:48.54775 golang.org/x/sync/errgroup.(*Group).Go.func1() 2022-12-06_11:05:48.54779 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67 2022-12-06_11:05:48.54783 created by golang.org/x/sync/errgroup.(*Group).Go 2022-12-06_11:05:48.54787 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d 2022-12-06_11:05:48.54791 2022-12-06_11:05:48.54794 goroutine 166 [select, 20 minutes]: 2022-12-06_11:05:48.54798 github.com/grafana/grafana/pkg/infra/usagestats.(*UsageStatsService).Run(0x0?, {0x2670b00, 0xc000339940}) 2022-12-06_11:05:48.54802 /var/cache/omnibus/src/grafana/pkg/infra/usagestats/service.go:63 +0x185 2022-12-06_11:05:48.54806 github.com/grafana/grafana/pkg/server.(*Server).Run.func1() 2022-12-06_11:05:48.54811 /var/cache/omnibus/src/grafana/pkg/server/server.go:177 +0x49 2022-12-06_11:05:48.54815 golang.org/x/sync/errgroup.(*Group).Go.func1() 2022-12-06_11:05:48.54819 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67 2022-12-06_11:05:48.54823 created by golang.org/x/sync/errgroup.(*Group).Go 2022-12-06_11:05:48.54827 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d 2022-12-06_11:05:48.54831 2022-12-06_11:05:48.54834 goroutine 167 [select, 20 minutes]: 2022-12-06_11:05:48.54838 github.com/grafana/grafana/pkg/services/provisioning.(*provisioningServiceImpl).Run(0xc0003a2600, {0x2670b00, 0xc000339940}) 2022-12-06_11:05:48.54842 /var/cache/omnibus/src/grafana/pkg/services/provisioning/provisioning.go:105 +0x212 2022-12-06_11:05:48.54846 github.com/grafana/grafana/pkg/server.(*Server).Run.func1() 2022-12-06_11:05:48.54850 /var/cache/omnibus/src/grafana/pkg/server/server.go:177 +0x49 2022-12-06_11:05:48.54854 golang.org/x/sync/errgroup.(*Group).Go.func1() 2022-12-06_11:05:48.54857 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:57 +0x67 2022-12-06_11:05:48.54861 created by golang.org/x/sync/errgroup.(*Group).Go 2022-12-06_11:05:48.54865 /root/go/pkg/mod/golang.org/x/sync@v0.0.0-20201207232520-09787c993a3a/errgroup/errgroup.go:54 +0x8d 2022-12-06_11:05:48.54870 2022-12-06_11:05:48.54874 goroutine 110 [select]: 2022-12-06_11:05:48.54878 github.com/grafana/grafana/pkg/plugins/backendplugin.restartKilledProcess({0x2670b00, 0xc000339940}, {0x2678f90, 0xc00020f0e0}) 2022-12-06_11:05:48.54882 /var/cache/omnibus/src/grafana/pkg/plugins/backendplugin/manager.go:442 +0xad 2022-12-06_11:05:48.54886 github.com/grafana/grafana/pkg/plugins/backendplugin.startPluginAndRestartKilledProcesses.func1({0x2670b00?, 0xc000339940?}, {0x2678f90?, 0xc00020f0e0?}) 2022-12-06_11:05:48.54890 /var/cache/omnibus/src/grafana/pkg/plugins/backendplugin/manager.go:430 +0x3b 2022-12-06_11:05:48.54894 created by github.com/grafana/grafana/pkg/plugins/backendplugin.startPluginAndRestartKilledProcesses 2022-12-06_11:05:48.54899 /var/cache/omnibus/src/grafana/pkg/plugins/backendplugin/manager.go:429 +0xcb 2022-12-06_11:05:48.54903 2022-12-06_11:05:48.54907 goroutine 47 [chan receive, 20 minutes]: 2022-12-06_11:05:48.54911 github.com/grafana/grafana/pkg/api.(*HTTPServer).Run.func1() 2022-12-06_11:05:48.54914 /var/cache/omnibus/src/grafana/pkg/api/http_server.go:141 +0x71 2022-12-06_11:05:48.54918 created by github.com/grafana/grafana/pkg/api.(*HTTPServer).Run 2022-12-06_11:05:48.54922 /var/cache/omnibus/src/grafana/pkg/api/http_server.go:138 +0x5be 2022-12-06_11:05:48.54926 2022-12-06_11:05:48.54930 goroutine 168 [select, 2 minutes]: 2022-12-06_11:05:48.54934 github.com/grafana/grafana/pkg/services/provisioning/dashboards.(*FileReader).pollChanges(0xc0003da3c0, {0x2670b00, 0xc0003db6c0}) 2022-12-06_11:05:48.54939 /var/cache/omnibus/src/grafana/pkg/services/provisioning/dashboards/file_reader.go:68 +0xa9 2022-12-06_11:05:48.54943 created by github.com/grafana/grafana/pkg/services/provisioning/dashboards.(*Provisioner).PollChanges 2022-12-06_11:05:48.54947 /var/cache/omnibus/src/grafana/pkg/services/provisioning/dashboards/dashboard.go:93 +0x45 2022-12-06_11:05:48.54950 2022-12-06_11:05:48.54954 goroutine 265 [IO wait]: 2022-12-06_11:05:48.54958 internal/poll.runtime_pollWait(0x7fce863f80e8, 0x72) 2022-12-06_11:05:48.54962 /usr/local/go/src/runtime/netpoll.go:302 +0x89 2022-12-06_11:05:48.54966 internal/poll.(*pollDesc).wait(0xc00056a580?, 0xc000527980?, 0x0) 2022-12-06_11:05:48.54969 /usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x32 2022-12-06_11:05:48.54973 internal/poll.(*pollDesc).waitRead(...) 2022-12-06_11:05:48.54977 /usr/local/go/src/internal/poll/fd_poll_runtime.go:88 2022-12-06_11:05:48.54982 internal/poll.(*FD).Read(0xc00056a580, {0xc000527980, 0x18c9, 0x18c9}) 2022-12-06_11:05:48.54987 /usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a 2022-12-06_11:05:48.54990 net.(*netFD).Read(0xc00056a580, {0xc000527980?, 0xc000527985?, 0x1e0?}) 2022-12-06_11:05:48.54994 /usr/local/go/src/net/fd_posix.go:55 +0x29 2022-12-06_11:05:48.54998 net.(*conn).Read(0xc0000c4d08, {0xc000527980?, 0xc000577820?, 0xc000f37318?}) 2022-12-06_11:05:48.55003 /usr/local/go/src/net/net.go:183 +0x45 2022-12-06_11:05:48.55007 crypto/tls.(*atLeastReader).Read(0xc000f7c768, {0xc000527980?, 0x0?, 0x21aeb00?}) 2022-12-06_11:05:48.55011 /usr/local/go/src/crypto/tls/conn.go:785 +0x3d 2022-12-06_11:05:48.55015 bytes.(*Buffer).ReadFrom(0xc000f373f8, {0x2657ee0, 0xc000f7c768}) 2022-12-06_11:05:48.55019 /usr/local/go/src/bytes/buffer.go:204 +0x98 2022-12-06_11:05:48.55023 crypto/tls.(*Conn).readFromUntil(0xc000f37180, {0x7fce84fcee60?, 0xc0003dac00}, 0x10c?) 2022-12-06_11:05:48.55027 /usr/local/go/src/crypto/tls/conn.go:807 +0xe5 2022-12-06_11:05:48.55030 crypto/tls.(*Conn).readRecordOrCCS(0xc000f37180, 0x0) 2022-12-06_11:05:48.55034 /usr/local/go/src/crypto/tls/conn.go:614 +0x116 2022-12-06_11:05:48.55038 crypto/tls.(*Conn).readRecord(...) 2022-12-06_11:05:48.55042 /usr/local/go/src/crypto/tls/conn.go:582 2022-12-06_11:05:48.55046 crypto/tls.(*Conn).Read(0xc000f37180, {0xc000725000, 0x1000, 0x1?}) 2022-12-06_11:05:48.55050 /usr/local/go/src/crypto/tls/conn.go:1285 +0x16f 2022-12-06_11:05:48.55053 net/http.(*persistConn).Read(0xc000223320, {0xc000725000?, 0xc0000a83c0?, 0xc000577d30?}) 2022-12-06_11:05:48.55057 /usr/local/go/src/net/http/transport.go:1929 +0x4e 2022-12-06_11:05:48.55061 bufio.(*Reader).fill(0xc000bd1c20) 2022-12-06_11:05:48.55065 /usr/local/go/src/bufio/bufio.go:106 +0x103 2022-12-06_11:05:48.55069 bufio.(*Reader).Peek(0xc000bd1c20, 0x1) 2022-12-06_11:05:48.55072 /usr/local/go/src/bufio/bufio.go:144 +0x5d 2022-12-06_11:05:48.55078 net/http.(*persistConn).readLoop(0xc000223320) 2022-12-06_11:05:48.55081 /usr/local/go/src/net/http/transport.go:2093 +0x1ac 2022-12-06_11:05:48.55085 created by net/http.(*Transport).dialConn 2022-12-06_11:05:48.55089 /usr/local/go/src/net/http/transport.go:1750 +0x173e 2022-12-06_11:05:48.55093 2022-12-06_11:05:48.55098 goroutine 239 [IO wait]: 2022-12-06_11:05:48.55102 internal/poll.runtime_pollWait(0x7fce863f7f08, 0x72) 2022-12-06_11:05:48.55106 /usr/local/go/src/runtime/netpoll.go:302 +0x89 2022-12-06_11:05:48.55110 internal/poll.(*pollDesc).wait(0xc0003fc100?, 0xc000d6a5e1?, 0x0) 2022-12-06_11:05:48.55114 /usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x32 2022-12-06_11:05:48.55117 internal/poll.(*pollDesc).waitRead(...) 2022-12-06_11:05:48.55123 /usr/local/go/src/internal/poll/fd_poll_runtime.go:88 2022-12-06_11:05:48.55127 internal/poll.(*FD).Read(0xc0003fc100, {0xc000d6a5e1, 0x1, 0x1}) 2022-12-06_11:05:48.55131 /usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a 2022-12-06_11:05:48.55135 net.(*netFD).Read(0xc0003fc100, {0xc000d6a5e1?, 0xc0008cfba0?, 0xc0008cfc28?}) 2022-12-06_11:05:48.55138 /usr/local/go/src/net/fd_posix.go:55 +0x29 2022-12-06_11:05:48.55142 net.(*conn).Read(0xc0000c46e8, {0xc000d6a5e1?, 0x0?, 0xc00053af01?}) 2022-12-06_11:05:48.55146 /usr/local/go/src/net/net.go:183 +0x45 2022-12-06_11:05:48.55150 net/http.(*connReader).backgroundRead(0xc000d6a5d0) 2022-12-06_11:05:48.55171 /usr/local/go/src/net/http/server.go:672 +0x3f 2022-12-06_11:05:48.55175 created by net/http.(*connReader).startBackgroundRead 2022-12-06_11:05:48.55180 /usr/local/go/src/net/http/server.go:668 +0xca 2022-12-06_11:05:48.55184 2022-12-06_11:05:48.55188 goroutine 175 [chan receive, 4 minutes]: 2022-12-06_11:05:48.55192 github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init.func1() 2022-12-06_11:05:48.55195 /var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:168 +0x34 2022-12-06_11:05:48.55199 created by github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init 2022-12-06_11:05:48.55202 /var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:166 +0x65 2022-12-06_11:05:48.55205 2022-12-06_11:05:48.55208 goroutine 176 [chan receive, 4 minutes]: 2022-12-06_11:05:48.55211 github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init.func1() 2022-12-06_11:05:48.55215 /var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:168 +0x34 2022-12-06_11:05:48.55218 created by github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init 2022-12-06_11:05:48.55221 /var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:166 +0x65 2022-12-06_11:05:48.55224 2022-12-06_11:05:48.55227 goroutine 177 [chan receive, 4 minutes]: 2022-12-06_11:05:48.55230 github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init.func1() 2022-12-06_11:05:48.55233 /var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:168 +0x34 2022-12-06_11:05:48.55237 created by github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init 2022-12-06_11:05:48.55240 /var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:166 +0x65 2022-12-06_11:05:48.55245 2022-12-06_11:05:48.55248 goroutine 194 [chan receive, 4 minutes]: 2022-12-06_11:05:48.55252 github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init.func1() 2022-12-06_11:05:48.55256 /var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:168 +0x34 2022-12-06_11:05:48.55260 created by github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init 2022-12-06_11:05:48.55264 /var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:166 +0x65 2022-12-06_11:05:48.55268 2022-12-06_11:05:48.55271 goroutine 195 [chan receive, 4 minutes]: 2022-12-06_11:05:48.55275 github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init.func1() 2022-12-06_11:05:48.55279 /var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:168 +0x34 2022-12-06_11:05:48.55283 created by github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init 2022-12-06_11:05:48.55287 /var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:166 +0x65 2022-12-06_11:05:48.55291 2022-12-06_11:05:48.55294 goroutine 196 [chan receive, 4 minutes]: 2022-12-06_11:05:48.55298 github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init.func1() 2022-12-06_11:05:48.55302 /var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:168 +0x34 2022-12-06_11:05:48.55306 created by github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init 2022-12-06_11:05:48.55310 /var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:166 +0x65 2022-12-06_11:05:48.55315 2022-12-06_11:05:48.55319 goroutine 197 [chan receive, 4 minutes]: 2022-12-06_11:05:48.55322 github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init.func1() 2022-12-06_11:05:48.55328 /var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:168 +0x34 2022-12-06_11:05:48.55332 created by github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init 2022-12-06_11:05:48.55335 /var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:166 +0x65 2022-12-06_11:05:48.55339 2022-12-06_11:05:48.55343 goroutine 198 [chan receive, 4 minutes]: 2022-12-06_11:05:48.55347 github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init.func1() 2022-12-06_11:05:48.55351 /var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:168 +0x34 2022-12-06_11:05:48.55357 created by github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init 2022-12-06_11:05:48.55362 /var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:166 +0x65 2022-12-06_11:05:48.55366 2022-12-06_11:05:48.55369 goroutine 199 [chan receive, 4 minutes]: 2022-12-06_11:05:48.55373 github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init.func1() 2022-12-06_11:05:48.55377 /var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:168 +0x34 2022-12-06_11:05:48.55382 created by github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init 2022-12-06_11:05:48.55387 /var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:166 +0x65 2022-12-06_11:05:48.55391 2022-12-06_11:05:48.55395 goroutine 200 [chan receive, 4 minutes]: 2022-12-06_11:05:48.55399 github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init.func1() 2022-12-06_11:05:48.55403 /var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:168 +0x34 2022-12-06_11:05:48.55408 created by github.com/grafana/grafana/pkg/api/avatar.(*Thunder).init 2022-12-06_11:05:48.55412 /var/cache/omnibus/src/grafana/pkg/api/avatar/avatar.go:166 +0x65 2022-12-06_11:05:48.55416 2022-12-06_11:05:48.55419 goroutine 266 [select]: 2022-12-06_11:05:48.55423 net/http.(*persistConn).writeLoop(0xc000223320) 2022-12-06_11:05:48.55427 /usr/local/go/src/net/http/transport.go:2392 +0xf5 2022-12-06_11:05:48.55431 created by net/http.(*Transport).dialConn 2022-12-06_11:05:48.55434 /usr/local/go/src/net/http/transport.go:1751 +0x1791 2022-12-06_11:05:48.60991 t=2022-12-06T12:05:48+0100 lvl=info msg="Starting Grafana" logger=server version=7.5.16 commit=c0e2ad126c branch=HEAD compiled=2022-05-19T12:39:41+0200 2022-12-06_11:05:48.61003 t=2022-12-06T12:05:48+0100 lvl=info msg="Config loaded from" logger=settings file=/var/opt/gitlab/grafana/conf/defaults.ini 2022-12-06_11:05:48.61007 t=2022-12-06T12:05:48+0100 lvl=info msg="Config loaded from" logger=settings file=/var/opt/gitlab/grafana/grafana.ini 2022-12-06_11:05:48.61014 t=2022-12-06T12:05:48+0100 lvl=info msg="Path Home" logger=settings path=/var/opt/gitlab/grafana 2022-12-06_11:05:48.61017 t=2022-12-06T12:05:48+0100 lvl=info msg="Path Data" logger=settings path=/var/opt/gitlab/grafana/data 2022-12-06_11:05:48.61022 t=2022-12-06T12:05:48+0100 lvl=info msg="Path Logs" logger=settings path=/var/log/gitlab/grafana 2022-12-06_11:05:48.61027 t=2022-12-06T12:05:48+0100 lvl=info msg="Path Plugins" logger=settings path=/var/opt/gitlab/grafana/data/plugins 2022-12-06_11:05:48.61032 t=2022-12-06T12:05:48+0100 lvl=info msg="Path Provisioning" logger=settings path=/var/opt/gitlab/grafana/provisioning 2022-12-06_11:05:48.61037 t=2022-12-06T12:05:48+0100 lvl=info msg="App mode production" logger=settings 2022-12-06_11:05:48.61195 t=2022-12-06T12:05:48+0100 lvl=info msg="Connecting to DB" logger=sqlstore dbtype=sqlite3 2022-12-06_11:05:48.61315 t=2022-12-06T12:05:48+0100 lvl=info msg="Starting DB migrations" logger=migrator 2022-12-06_11:05:48.62625 t=2022-12-06T12:05:48+0100 lvl=info msg="migrations completed" logger=migrator performed=0 skipped=279 duration=1.490749ms 2022-12-06_11:05:48.62919 t=2022-12-06T12:05:48+0100 lvl=info msg="Starting plugin search" logger=plugins 2022-12-06_11:05:48.69082 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.690741+01:00","queryType":"exponential_heatmap_bucket_data"} 2022-12-06_11:05:48.69100 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.690971+01:00","queryType":"linear_heatmap_bucket_data"} 2022-12-06_11:05:48.69114 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.691086+01:00","queryType":"random_walk"} 2022-12-06_11:05:48.69131 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.691282+01:00","queryType":"predictable_pulse"} 2022-12-06_11:05:48.69141 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.691385+01:00","queryType":"predictable_csv_wave"} 2022-12-06_11:05:48.69151 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.691484+01:00","queryType":"random_walk_table"} 2022-12-06_11:05:48.69161 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.691587+01:00","queryType":"slow_query"} 2022-12-06_11:05:48.69170 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.691677+01:00","queryType":"no_data_points"} 2022-12-06_11:05:48.69180 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.691777+01:00","queryType":"datapoints_outside_range"} 2022-12-06_11:05:48.69190 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.691874+01:00","queryType":"manual_entry"} 2022-12-06_11:05:48.69199 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.691965+01:00","queryType":"csv_metric_values"} 2022-12-06_11:05:48.69208 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.692057+01:00","queryType":"streaming_client"} 2022-12-06_11:05:48.69217 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.692148+01:00","queryType":"live"} 2022-12-06_11:05:48.69227 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.692243+01:00","queryType":"grafana_api"} 2022-12-06_11:05:48.69236 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.692337+01:00","queryType":"arrow"} 2022-12-06_11:05:48.69245 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.692428+01:00","queryType":"annotations"} 2022-12-06_11:05:48.69254 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.692521+01:00","queryType":"table_static"} 2022-12-06_11:05:48.69266 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.692612+01:00","queryType":"random_walk_with_error"} 2022-12-06_11:05:48.69275 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.692726+01:00","queryType":"server_error_500"} 2022-12-06_11:05:48.69284 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.692819+01:00","queryType":"logs"} 2022-12-06_11:05:48.69296 {"@level":"debug","@message":"datasource: registering query type handler","@timestamp":"2022-12-06T12:05:48.692909+01:00","queryType":"node_graph"} 2022-12-06_11:05:48.69305 {"@level":"debug","@message":"datasource: registering query type fallback handler","@timestamp":"2022-12-06T12:05:48.693025+01:00"} 2022-12-06_11:05:48.70735 t=2022-12-06T12:05:48+0100 lvl=info msg="HTTP Server Listen" logger=http.server address=127.0.0.1:3000 protocol=http subUrl=/-/grafana socket=
Results of GitLab environment info
Expand for output related to GitLab environment info
System information System: Debian 11 Current User: git Using RVM: no Ruby Version: 2.7.6p219 Gem Version: 3.1.6 Bundler Version:2.3.15 Rake Version: 13.0.6 Redis Version: 6.2.7 Sidekiq Version:6.5.7 Go Version: unknown GitLab information Version: 15.6.2 Revision: 2d7e4701985 Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: PostgreSQL DB Version: 12.12 URL: https://gitlab.domain.redacted.org HTTP Clone URL: https://gitlab.domain.redacted.org/some-group/some-project.git SSH Clone URL: git@gitlab.domain.redacted.org:some-group/some-project.git Using LDAP: no Using Omniauth: yes Omniauth Providers: google_oauth2 GitLab Shell Version: 14.13.0 Repository storage paths: - default: /srv/gitlab/git-data/repositories GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell
Results of GitLab application Check
Expand for output related to the GitLab application check
Checking GitLab subtasks ...Checking GitLab Shell ...
GitLab Shell: ... GitLab Shell version >= 14.13.0 ? ... OK (14.13.0) Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Internal API available: OK Redis available via internal API: OK gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Gitaly ...
Gitaly: ... default ... OK
Checking Gitaly ... Finished
Checking Sidekiq ...
Sidekiq: ... Running? ... yes Number of Sidekiq processes (cluster/worker) ... 1/1
Checking Sidekiq ... Finished
Checking Incoming Email ...
Incoming Email: ... Reply by email is disabled in config/gitlab.yml
Checking Incoming Email ... Finished
Checking LDAP ...
LDAP: ... LDAP is disabled in config/gitlab.yml
Checking LDAP ... Finished
Checking GitLab App ...
Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... yes Uploads directory tmp has correct permissions? ... yes Systemd unit files or init script exist? ... skipped (omnibus-gitlab has neither init script nor systemd units) Systemd unit files or init script up-to-date? ... skipped (omnibus-gitlab has neither init script nor systemd units) Projects have namespace: ... 3/1 ... yes 3/2 ... yes 4/3 ... yes 4/4 ... yes 8/5 ... yes 4/6 ... yes 10/7 ... yes 7/8 ... yes 12/13 ... yes 7/15 ... yes 12/16 ... yes 12/17 ... yes 10/18 ... yes 10/20 ... yes 22/21 ... yes 15/22 ... yes 3/23 ... yes 23/24 ... yes 10/25 ... yes 10/26 ... yes 10/27 ... yes 10/28 ... yes 3/29 ... yes 23/30 ... yes 23/31 ... yes 27/36 ... yes 23/37 ... yes 23/38 ... yes 23/39 ... yes 10/40 ... yes 23/42 ... yes 10/43 ... yes 8/45 ... yes 10/46 ... yes 10/47 ... yes 10/48 ... yes 22/50 ... yes 87/52 ... yes 87/53 ... yes 87/54 ... yes 87/55 ... yes 87/56 ... yes 87/57 ... yes 4/58 ... yes 34/59 ... yes 22/60 ... yes 23/61 ... yes 35/62 ... yes 35/63 ... yes 35/64 ... yes 35/65 ... yes 35/66 ... yes 35/67 ... yes 35/68 ... yes 8/69 ... yes 8/70 ... yes 23/71 ... yes 23/72 ... yes 22/73 ... yes 38/74 ... yes 38/75 ... yes 38/76 ... yes 38/77 ... yes 38/78 ... yes 35/79 ... yes 9/80 ... yes 23/81 ... yes 23/83 ... yes 23/84 ... yes 23/85 ... yes 10/86 ... yes 36/87 ... yes 12/88 ... yes 87/89 ... yes 87/90 ... yes 22/91 ... yes 35/92 ... yes 22/93 ... yes 22/94 ... yes 22/96 ... yes 22/97 ... yes 10/98 ... yes 35/99 ... yes 38/100 ... yes 7/101 ... yes 10/102 ... yes 27/103 ... yes 23/104 ... yes 23/105 ... yes 22/106 ... yes 10/107 ... yes 86/108 ... yes 86/109 ... yes 86/110 ... yes 86/111 ... yes 86/112 ... yes 86/113 ... yes 86/115 ... yes 34/116 ... yes 86/117 ... yes 23/118 ... yes 23/119 ... yes 45/120 ... yes 4/121 ... yes 45/122 ... yes 2/123 ... yes 45/124 ... yes 45/125 ... yes 45/126 ... yes 2/127 ... yes 23/128 ... yes 8/129 ... yes 22/130 ... yes 48/133 ... yes 48/134 ... yes 50/138 ... yes 45/139 ... yes 48/140 ... yes 12/142 ... yes 45/143 ... yes 7/144 ... yes 35/146 ... yes 35/147 ... yes 35/148 ... yes 9/157 ... yes 9/161 ... yes 87/163 ... yes 4/164 ... yes 23/166 ... yes 23/167 ... yes 10/168 ... yes 10/169 ... yes 23/170 ... yes 8/171 ... yes 8/172 ... yes 8/173 ... yes 8/174 ... yes 34/175 ... yes 31/176 ... yes 23/177 ... yes 45/178 ... yes 45/179 ... yes 45/180 ... yes 34/181 ... yes 56/182 ... yes 57/183 ... yes 86/185 ... yes 57/186 ... yes 45/187 ... yes 23/188 ... yes 48/189 ... yes 57/190 ... yes 60/191 ... yes 50/192 ... yes 60/193 ... yes 23/194 ... yes 57/195 ... yes 60/196 ... yes 60/197 ... yes 8/198 ... yes 56/199 ... yes 22/200 ... yes 23/201 ... yes 61/202 ... yes 45/203 ... yes 12/204 ... yes 45/205 ... yes 45/206 ... yes 23/207 ... yes 23/208 ... yes 23/209 ... yes 23/210 ... yes 87/211 ... yes 55/212 ... yes 38/213 ... yes 39/214 ... yes 2/216 ... yes 75/217 ... yes 35/218 ... yes 87/219 ... yes 56/220 ... yes 27/221 ... yes 39/222 ... yes 45/223 ... yes 45/224 ... yes 87/225 ... yes 35/226 ... yes 35/227 ... yes 35/228 ... yes 70/229 ... yes 64/230 ... yes 64/231 ... yes 70/233 ... yes 61/234 ... yes 23/235 ... yes 86/236 ... yes 64/237 ... yes 39/238 ... yes 70/239 ... yes 23/240 ... yes 35/241 ... yes 35/242 ... yes 70/243 ... yes 39/244 ... yes 57/245 ... yes 23/246 ... yes 23/247 ... yes 23/248 ... yes 70/250 ... yes 70/251 ... yes 35/252 ... yes 39/253 ... yes 39/255 ... yes 61/256 ... yes 61/257 ... yes 71/258 ... yes 87/259 ... yes 72/260 ... yes 61/261 ... yes 70/262 ... yes 55/263 ... yes 48/264 ... yes 38/265 ... yes 48/266 ... yes 76/267 ... yes 34/268 ... yes 76/269 ... yes 70/270 ... yes 75/271 ... yes 48/273 ... yes 48/274 ... yes 23/275 ... yes 38/276 ... yes 23/277 ... yes 45/278 ... yes 75/279 ... yes 75/280 ... yes 76/281 ... yes 76/282 ... yes 76/283 ... yes 76/284 ... yes 10/285 ... yes 38/286 ... yes 413/304 ... yes 76/305 ... yes 76/306 ... yes 55/307 ... yes 10/308 ... yes 60/313 ... yes 57/314 ... yes 57/315 ... yes 70/316 ... yes 76/317 ... yes 76/318 ... yes 413/319 ... yes 23/320 ... yes 56/321 ... yes 39/322 ... yes 76/323 ... yes 70/324 ... yes 55/325 ... yes 2/326 ... yes 2/327 ... yes 2/328 ... yes 2/329 ... yes 45/330 ... yes 10/331 ... yes 84/332 ... yes 86/333 ... yes 7/334 ... yes 45/335 ... yes 76/336 ... yes 55/337 ... yes 48/338 ... yes 87/339 ... yes 413/340 ... yes 59/341 ... yes 70/342 ... yes 75/343 ... yes 55/344 ... yes 39/345 ... yes 45/346 ... yes 35/347 ... yes 70/348 ... yes 86/349 ... yes 76/350 ... yes 23/351 ... yes 2/352 ... yes 27/353 ... yes 55/354 ... yes 70/355 ... yes 48/356 ... yes 70/357 ... yes 70/358 ... yes 76/359 ... yes 34/360 ... yes 2/361 ... yes 8/362 ... yes 2/363 ... yes 23/364 ... yes 23/365 ... yes 408/366 ... yes 76/367 ... yes 35/368 ... yes 418/369 ... yes 61/370 ... yes 4/371 ... yes 45/372 ... yes 50/373 ... yes 50/374 ... yes 50/375 ... yes 23/376 ... yes 38/377 ... yes 38/378 ... yes 23/379 ... yes 22/380 ... yes 38/381 ... yes 75/382 ... yes 45/383 ... yes 45/384 ... yes 45/385 ... yes 75/386 ... yes 76/387 ... yes 35/388 ... yes 35/389 ... yes 70/390 ... yes 8/391 ... yes 8/392 ... yes 12/393 ... yes 76/394 ... yes 23/395 ... yes 76/396 ... yes 2/397 ... yes 2/398 ... yes 76/400 ... yes 46/401 ... yes 87/402 ... yes 456/403 ... yes 449/404 ... yes 70/405 ... yes 449/406 ... yes 12/408 ... yes 10/409 ... yes 70/410 ... yes 76/411 ... yes 76/412 ... yes 38/413 ... yes 84/414 ... yes 84/415 ... yes 70/416 ... yes 27/417 ... yes 84/418 ... yes 70/419 ... yes 86/420 ... yes 76/421 ... yes 76/422 ... yes 75/423 ... yes 485/424 ... yes 12/425 ... yes 489/426 ... yes 87/427 ... yes 87/428 ... yes 87/429 ... yes 50/430 ... yes 50/431 ... yes 48/432 ... yes 48/433 ... yes 48/434 ... yes 48/435 ... yes 70/436 ... yes 76/437 ... yes 35/438 ... yes 39/439 ... yes 87/440 ... yes 60/441 ... yes 60/442 ... yes 76/443 ... yes 76/444 ... yes 75/445 ... yes 45/446 ... yes 522/447 ... yes 35/448 ... yes 76/449 ... yes 70/450 ... yes 39/451 ... yes 70/452 ... yes 38/453 ... yes 76/454 ... yes 87/455 ... yes 70/456 ... yes 87/457 ... yes 23/458 ... yes 76/459 ... yes 86/461 ... yes 81/462 ... yes Redis version >= 6.0.0? ... yes Ruby version >= 2.7.2 ? ... yes (2.7.6) Git user has default SSH configuration? ... yes Active users: ... 52 Is authorized keys file accessible? ... yes GitLab configured to store new projects in hashed storage? ... yes All projects are in hashed storage? ... yes
Checking GitLab App ... Finished
Checking GitLab subtasks ... Finished
Additional info
InfluxDB instance I am attempting to use is in local network and the connection has been functional prior to the update. When creating a temporary instance of Grafana in a Docker container the connection can be established without issues, including when creating the container on the same machine that GitLab instance is running. I am eager to provide any additional information should it be required.