Add db to trace insert query
This has gone unnoticed because we mainly set /tracing
at the end of our clickhouse-dsn
env vars.
Found this in our staging cluster:
2024-08-20T20:32:50.208Z info exporterhelper/retry_sender.go:118 Exporting failed. Will retry the request after interval. {"kind": "exporter", "data_type": "traces", "name": "gitlabobservability", "error": "preparing traces batch for clickhouse backend: code: 60, message: Table default.gl_traces_main does not exist. Maybe you meant clickhouse_test.ch_gl_traces_main?", "interval": "35.135227511s"}
2024-08-20T20:33:25.368Z error exporterhelper/common.go:296 Exporting failed. Rejecting data. {"kind": "exporter", "data_type": "traces", "name": "gitlabobservability", "error": "no more retries left: preparing traces batch for clickhouse backend: code: 60, message: Table default.gl_traces_main does not exist. Maybe you meant clickhouse_test.ch_gl_traces_main?", "rejected_items": 20}
go.opentelemetry.io/collector/exporter/exporterhelper.(*baseExporter).send
/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.101.0/exporterhelper/common.go:296
go.opentelemetry.io/collector/exporter/exporterhelper.NewTracesRequestExporter.func1
/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.101.0/exporterhelper/traces.go:134
go.opentelemetry.io/collector/consumer.ConsumeTracesFunc.ConsumeTraces
/go/pkg/mod/go.opentelemetry.io/collector/consumer@v0.102.1/traces.go:25
go.opentelemetry.io/collector/processor/batchprocessor.(*batchTraces).export
/go/pkg/mod/go.opentelemetry.io/collector/processor/batchprocessor@v0.101.0/batch_processor.go:414
go.opentelemetry.io/collector/processor/batchprocessor.(*shard).sendItems
/go/pkg/mod/go.opentelemetry.io/collector/processor/batchprocessor@v0.101.0/batch_processor.go:261
go.opentelemetry.io/collector/processor/batchprocessor.(*shard).startLoop
/go/pkg/mod/go.opentelemetry.io/collector/processor/batchprocessor@v0.101.0/batch_processor.go:223
2024-08-20T20:33:25.368Z warn batchprocessor@v0.101.0/batch_processor.go:263 Sender failed {"kind": "processor", "name": "batch", "pipeline": "traces", "error": "no more retries left: preparing traces batch for clickhouse backend: code: 60, message: Table default.gl_traces_main does not exist. Maybe you meant clickhouse_test.ch_gl_traces_main?"}
Checking our DSN, we don't have /tracing
attached. This resolves the issue by setting the DB during the SQL, irrespective of the DSN specified DB.