Skip to content

Remove clickhouse node dependency due to potential security issues.

Robert Hunt requested to merge replace-clickhouse-node-dependency into main

What does this MR do and why?

Removes clickhouse node dependency due to potential security issues [1, 2].

Replaced with Clickhouse-controlled node package and update code to use its methods instead.

Due to the fact that we set the values for DevKit/Analytics stack, I don't think these vulnerabilities are currently exploitable but it's better to not have them there at all.

How to set up and validate locally

  1. Pull this branch
  2. Add a pre-aggregation to a schema, e.g. 
    Index: cube/schema/TrackedEvents.js
===================================================================
diff --git a/cube/schema/TrackedEvents.js b/cube/schema/TrackedEvents.js
--- a/cube/schema/TrackedEvents.js	(revision Staged)
+++ b/cube/schema/TrackedEvents.js	(date 1700484054755)
@@ -141,5 +141,50 @@
       type: `string`
     },
   },
+  pre_aggregations: {
+    pageViews: {
+      indexes: {
+        index: {
+          columns: [event],
+        },
+      },
+      external: false,
+      refresh_key: {
+        every: `1 hour`
+      },
+      measures: [
+        TrackedEvents.pageViewsCount
+      ],
+      dimensions: [
+        TrackedEvents.event,
+        TrackedEvents.derivedTstamp
+      ],
+      timeDimension: TrackedEvents.derivedTstamp,
+      granularity: `second`,
+      partition_granularity: `day`
+    },
+
+    events: {
+      indexes: {
+        index: {
+          columns: [event],
+        },
+      },
+      external: false,
+      refresh_key: {
+        every: `1 hour`,
+      },
+      measures: [
+        TrackedEvents.count
+      ],
+      dimensions: [
+        TrackedEvents.event,
+        TrackedEvents.derivedTstamp
+      ],
+      timeDimension: TrackedEvents.derivedTstamp,
+      granularity: `second`,
+      partition_granularity: `day`
+    }
+  },
   dataSource: `default`
 });
  3. Restart your cube container
  4. Verify that pre-aggregations are run for your local tables without throwing errors.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Merge request reports