@bohdanpk,

We ran into this a while back. None of these solutions are good. Simply remove the nullability from the CH table which should solve the issue.

Context: traversal_ids column is under backfill, eventually it will be NOT NULL. We don't have other array columns which are nullable.

@ahegyi Hey! I am not familiar with how siphon works this is why I hesitated to put this AI-investigation under an issue, because it might have been just a misconfiguration on our part (or hallucination of an agent).

Simply remove the nullability from the CH table which should solve the issue.

We don't have nullable arrays in our tables that we use for siphon, in fact, it is impossible to have nullable arrays. As far as I understand, siphon tried to insert a TOASTed array column as a NULL, which is impossible for Clickhouse.

Gotcha, we'll look into this and cover it with a test case!

Looking at the MR, I believe I thought about this 😅

  	// For INSERT and UPDATE we will have all columns available.
  	// There is a chance where an array column is set to NULL which is not
  	// allowed in ClickHouse. Here we need to set the array value to an empty array.

But I can take a look the test cases included to reproduce this

Reproduced and fixed via gitlab-org/analytics-section/siphon!275 (merged)