Skip to content

Add dynamic schema

Max Woolf requested to merge mw/dynamic-cube-schema into main

Overview

  • Adds some changes to the Cube schema to facilitate multiple databases.
  • It is likely we'll decide to store each project's events in its own Clickhouse database.
  • This MR adds a Jitsu schema which reads events from the jitsu table on an individual database passed through to cube's security context object.
  • Also adds some minor changes to the README to help its use.

Note: This change is open to SQL injection (passing another table in to the security context!) and has not implemented any form of sensible authentication yet! (That's coming next...)

How to use

  • In Cube, alter the security context of a query to the following variables where:
  • iat is a Unix timestamp set to later today
  • exp is a Unix timestamp somewhere in the future
  • appId is the name of a Clickhouse database that contains data streamed to it from Jitsu in to a table named jitsu.
  • Screenshot_2022-08-22_at_14.25.12
  • Make some regular queries to check that it works, changing appId to valid, and invalid values to check that it switches databases as expected when querying.

Will this break my devkit?

Yes, initially. Once this hits main, you'll need to:

  1. Rename your Clickhouse database to someproject_1000.jitsu
  2. In the cube playground, use security contexts and add "appId": "someproject_1000" to it.
  3. Update your Jitsu setup to pass data to that new database.
Edited by Robert Hunt

Merge request reports