Update the Cube proxy query param to Hash type to fix POST API calls (!104577) · Merge requests · GitLab.org / GitLab

Robert Hunt requested to merge fix-cube-proxy-post-requests into master Nov 21, 2022

What does this MR do and why?

In Update analytics cube proxy to use POST (!103330 - merged) I changed the Cube proxy from GET to POST. However, I forgot to update the query params type from String to Hash.

This resulted in frontend queries returning query is invalid error messages. The specs should have caught this issue, but I forgot to update the specs to stop using .to_json. The API endpoint is behind a default-off feature flag and is not in use by our own code just yet, which makes validating difficult.

This MR fixes the param and updates the specs to reflect what is sent to the API endpoint. It also updates the response code to 200 because Cube requires it 🙈.

Screenshots or screen recordings

N/A

How to set up and validate locally

Make sure you are on at least GitLab Premium.

Testing this MR specifically

Enable the feature flag: echo "Feature.enable(:cube_api_proxy)" | rails c.
Set up the devkit and start it running using docker-compose up.
Visit admin settings and make sure that the settings have been set: /admin/application_settings/general#js-product-analytics-settings
As an authenticated user who has developer+ access to a particular project, make the following API call:

POST /api/v4/projects/YOUR_PROJECT_ID/product_analytics/request/load HTTP/1.1
Host: YOUR_HOST
PRIVATE-TOKEN: YOUR_TOKEN_HERE
Content-Type: application/json

{
  "query": {
    "dimensions": [
      "Jitsu.docPath"
    ],
    "order": [
      [
        "Jitsu.count",
        "desc"
      ],
      [
        "Jitsu.docPath",
        "desc"
      ],
      [
        "Jitsu.utcTime",
        "asc"
      ]
    ],
    "measures": [
      "Jitsu.count"
    ],
    "timeDimensions": [
      {
        "dimension": "Jitsu.utcTime",
        "granularity": "day",
        "dateRange": "This year"
      }
    ],
    "limit": 23
  },
  "queryType": "multi"
}

Expect it to respond with a 201

Testing with a Cube JS graph (for how it will be used in future, only test if you **really** want to)

Set up the devkit and start it running using docker-compose up.
Open the rails console: rails c
Enable the feature flag: Feature.enable(:cube_api_proxy).
Visit admin settings and make sure that the settings have been set: /admin/application_settings/general#js-product-analytics-settings
Find a project on the console: project = Project.find(PROJECT_ID)
Run ProductAnalytics::InitializeStackService.new(container: project).execute
Ensure that a sidekiq job was enqueued and in the Jitsu configurator the new API key is created.
Copy the JS API key and add the JS SDK to a local page to start generating data.
Apply the following patch to add Cube to the frontend:

Index: config/webpack.config.js
<+>UTF-8
===================================================================
diff --git a/config/webpack.config.js b/config/webpack.config.js
--- a/config/webpack.config.js	(revision Staged)
+++ b/config/webpack.config.js	(date 1667833878228)
@@ -306,6 +306,11 @@
         test: /\.mjs$/,
         use: [],
       },
+      {
+        test: /(@cubejs-client\/vue).*\.(js)?$/,
+        include: /node_modules/,
+        loader: 'babel-loader',
+      },
       WEBPACK_USE_ESBUILD_LOADER && {
         test: /\.js$/,
         exclude: (modulePath) =>
Index: ee/app/assets/javascripts/product_analytics/dashboards/components/widgets/cube_line_chart.vue
<+>UTF-8
===================================================================
diff --git a/ee/app/assets/javascripts/product_analytics/dashboards/components/widgets/cube_line_chart.vue b/ee/app/assets/javascripts/product_analytics/dashboards/components/widgets/cube_line_chart.vue
--- a/ee/app/assets/javascripts/product_analytics/dashboards/components/widgets/cube_line_chart.vue	(revision Staged)
+++ b/ee/app/assets/javascripts/product_analytics/dashboards/components/widgets/cube_line_chart.vue	(date 1667841400276)
@@ -1,8 +1,40 @@
 <script>
-import { s__ } from '~/locale';
+import { QueryRenderer } from '@cubejs-client/vue';
+import { GlLineChart } from '@gitlab/ui/dist/charts';
+import { CubejsApi, HttpTransport } from '@cubejs-client/core';
+import csrf from '~/lib/utils/csrf';
+
+const cubejsApi = new CubejsApi('1', {
+  transport: new HttpTransport({
+    apiUrl: '/api/v4/projects/1/product_analytics/request',
+    method: 'POST',
+    headers: {
+      [csrf.headerKey]: csrf.token,
+      'X-Requested-With': 'XMLHttpRequest',
+    },
+    credentials: 'same-origin',
+  }),
+});
+
+const overTimeQueries = {
+  users: {
+    measures: ['Jitsu.count'],
+    timeDimensions: [
+      {
+        granularity: 'day',
+        dimension: 'Jitsu.utcTime',
+      },
+    ],
+    dimensions: ['Jitsu.eventType'],
+  },
+};
 
 export default {
   name: 'CubeLineChart',
+  components: {
+    GlLineChart,
+    QueryRenderer,
+  },
   props: {
     data: {
       type: Object,
@@ -20,12 +52,57 @@
       default: () => ({}),
     },
   },
-  i18n: {
-    content: s__('ProductAnalytics|Widgets content'),
+  data() {
+    return { cubejsApi, query: this.withFilters(overTimeQueries.users, 'day') };
+  },
+  methods: {
+    series(resultSet) {
+      if (!resultSet) {
+        return [];
+      }
+
+      const seriesNames = resultSet?.seriesNames();
+      const pivot = resultSet?.chartPivot();
+      const series = [];
+
+      seriesNames.forEach((e) => {
+        const data = pivot.map((p) => [p.x, p[e.key]]);
+        series.push({
+          name: e.title,
+          data,
+        });
+      });
+
+      return series;
+    },
+    withFilters(query, granularity) {
+      const newQuery = {
+        ...query,
+      };
+
+      newQuery.timeDimensions = [
+        {
+          dimension: `Jitsu.utcTime`,
+          dateRange: [new Date(2022, 8), new Date(2022, 10)],
+          granularity: granularity || null,
+        },
+      ];
+
+      return newQuery;
+    },
   },
 };
 </script>
 
 <template>
-  <p>{{ $options.i18n.content }}</p>
+  <query-renderer :cubejs-api="cubejsApi" :query="query">
+    <template #default="{ resultSet }">
+      <gl-line-chart
+        :data="series(resultSet)"
+        :option="chartOptions"
+        :show-legend="!customizations.hideLegend"
+        responsive
+      />
+    </template>
+  </query-renderer>
 </template>
Index: jest.config.base.js
<+>UTF-8
===================================================================
diff --git a/jest.config.base.js b/jest.config.base.js
--- a/jest.config.base.js	(revision Staged)
+++ b/jest.config.base.js	(date 1667833913621)
@@ -153,6 +153,7 @@
     'dateformat',
     'lowlight',
     'vscode-languageserver-types',
+    '@cubejs-client/vue',
     ...gfmParserDependencies,
   ];
 
Index: package.json
<+>UTF-8
===================================================================
diff --git a/package.json b/package.json
--- a/package.json	(revision Staged)
+++ b/package.json	(date 1667833913631)
@@ -51,6 +51,8 @@
     "@babel/core": "^7.18.5",
     "@babel/preset-env": "^7.18.2",
     "@codesandbox/sandpack-client": "^1.2.2",
+    "@cubejs-client/core": "^0.31.0",
+    "@cubejs-client/vue": "^0.31.0",
     "@gitlab/at.js": "1.5.7",
     "@gitlab/favicon-overlay": "2.0.0",
     "@gitlab/svgs": "3.7.0",
Index: yarn.lock
<+>UTF-8
===================================================================
diff --git a/yarn.lock b/yarn.lock
--- a/yarn.lock	(revision Staged)
+++ b/yarn.lock	(date 1667834186006)
@@ -1041,6 +1041,27 @@
   resolved "https://registry.yarnpkg.com/@csstools/selector-specificity/-/selector-specificity-2.0.1.tgz#b6b8d81780b9a9f6459f4bfe9226ac6aefaefe87"
   integrity sha512-aG20vknL4/YjQF9BSV7ts4EWm/yrjagAN7OWBNmlbEOUiu0llj4OGrFoOKK3g2vey4/p2omKCoHrWtPxSwV3HA==
 
+"@cubejs-client/core@^0.31.0", "@cubejs-client/core@^0.31.9":
+  version "0.31.9"
+  resolved "https://registry.yarnpkg.com/@cubejs-client/core/-/core-0.31.9.tgz#5225a322a90f2dbdd3342e6b3b3ecc7abbc373ef"
+  integrity sha512-AXrAVdAcMN+WLyhHtXYMRQ+YfynUKO5zUEUORdWWPjqXf8zJl1wltLWQHlIAFs43bwjHopOTncf6sI4o7tTNAQ==
+  dependencies:
+    core-js "^3.6.5"
+    cross-fetch "^3.0.2"
+    dayjs "^1.10.4"
+    ramda "^0.27.0"
+    url-search-params-polyfill "^7.0.0"
+    uuid "^8.3.2"
+
+"@cubejs-client/vue@^0.31.0":
+  version "0.31.9"
+  resolved "https://registry.yarnpkg.com/@cubejs-client/vue/-/vue-0.31.9.tgz#19a68534589d2063539e0c6c0b03ae262d545044"
+  integrity sha512-p0s6dfefNMoywAk6PjbtHYTO6bH1wptu1c/wkjX7yl0r0KeJDHFvJX4mtWHbUmysxERgEfrgYzw0e3J2Nt8KeQ==
+  dependencies:
+    "@cubejs-client/core" "^0.31.9"
+    core-js "^3.6.5"
+    ramda "^0.27.0"
+
 "@discoveryjs/json-ext@^0.5.0":
   version "0.5.6"
   resolved "https://registry.yarnpkg.com/@discoveryjs/json-ext/-/json-ext-0.5.6.tgz#d5e0706cf8c6acd8c6032f8d54070af261bbbb2f"
@@ -3903,7 +3924,7 @@
   resolved "https://registry.yarnpkg.com/core-js-pure/-/core-js-pure-3.6.5.tgz#c79e75f5e38dbc85a662d91eea52b8256d53b813"
   integrity sha512-lacdXOimsiD0QyNf9BC/mxivNJ/ybBGJXQFKzRekp1WTHoVUWsUHEn+2T8GJAzzIhyOuXA+gOxCVN3l+5PLPUA==
 
-core-js@^3.26.0:
+core-js@^3.26.0, core-js@^3.6.5:
   version "3.26.0"
   resolved "https://registry.yarnpkg.com/core-js/-/core-js-3.26.0.tgz#a516db0ed0811be10eac5d94f3b8463d03faccfe"
   integrity sha512-+DkDrhoR4Y0PxDz6rurahuB+I45OsEUv8E1maPTB6OuHRohMMcznBq9TMpdpDMm/hUPob/mJJS3PqgbHpMTQgw==
@@ -3988,6 +4009,13 @@
   dependencies:
     jquery ">= 1.9.1"
 
+cross-fetch@^3.0.2:
+  version "3.1.5"
+  resolved "https://registry.yarnpkg.com/cross-fetch/-/cross-fetch-3.1.5.tgz#e1389f44d9e7ba767907f7af8454787952ab534f"
+  integrity sha512-lvb1SBsI0Z7GDwmuid+mU3kWVBwTVUbe7S0H52yaaAdQOXq2YktTCZdlAcNKFzE6QtRz0snpw9bNiPeOIkkQvw==
+  dependencies:
+    node-fetch "2.6.7"
+
 cross-spawn@^6.0.5:
   version "6.0.5"
   resolved "https://registry.yarnpkg.com/cross-spawn/-/cross-spawn-6.0.5.tgz#4a5ec7c64dfae22c3a14124dbacdee846d80cbc4"
@@ -4686,6 +4714,11 @@
   resolved "https://registry.yarnpkg.com/dateformat/-/dateformat-5.0.1.tgz#60a27a2deb339f888ba4532f533e25ac73ca3d19"
   integrity sha512-DrcKxOW2am3mtqoJwBTK3OlWcF0QSk1p8diEWwpu3Mf//VdURD7XVaeOV738JvcaBiFfm9o2fisoMhiJH0aYxg==
 
+dayjs@^1.10.4:
+  version "1.11.6"
+  resolved "https://registry.yarnpkg.com/dayjs/-/dayjs-1.11.6.tgz#2e79a226314ec3ec904e3ee1dd5a4f5e5b1c7afb"
+  integrity sha512-zZbY5giJAinCG+7AGaw0wIhNZ6J8AhWuSXKvuc1KAyMiRsvGQWqh4L+MomvhdAYjN+lqvVCMq1I41e3YHvXkyQ==
+
 de-indent@^1.0.2:
   version "1.0.2"
   resolved "https://registry.yarnpkg.com/de-indent/-/de-indent-1.0.2.tgz#b2038e846dc33baa5796128d0804b455b8c1e21d"
@@ -9259,7 +9292,7 @@
   resolved "https://registry.yarnpkg.com/node-domexception/-/node-domexception-1.0.0.tgz#6888db46a1f71c0b76b3f7555016b63fe64766e5"
   integrity sha512-/jKZoMpw0F8GRwl4/eLROPA3cfcXtLApP0QzLmUT/HuPCZWyB7IY9ZrMeKw2O/nFIqPQB3PVM9aYm0F312AXDQ==
 
-node-fetch@^2.6.1, node-fetch@^2.6.7:
+node-fetch@2.6.7, node-fetch@^2.6.1, node-fetch@^2.6.7:
   version "2.6.7"
   resolved "https://registry.yarnpkg.com/node-fetch/-/node-fetch-2.6.7.tgz#24de9fba827e3b4ae44dc8b20256a379160052ad"
   integrity sha512-ZjMPFEfVx5j+y2yF35Kzx5sF7kDzxuDj6ziH4FFbOp87zKDZNx8yExJIb05OGF4Nlt9IHFIMBkRl41VdvcNdbQ==
@@ -10292,6 +10325,11 @@
   resolved "https://registry.yarnpkg.com/quick-lru/-/quick-lru-4.0.1.tgz#5b8878f113a58217848c6482026c73e1ba57727f"
   integrity sha512-ARhCpm70fzdcvNQfPoy49IaanKkTlRWF2JMzqhcJbhSFRZv7nPTvZJdcY7301IPmvW+/p0RgIWnQDLJxifsQ7g==
 
+ramda@^0.27.0:
+  version "0.27.2"
+  resolved "https://registry.yarnpkg.com/ramda/-/ramda-0.27.2.tgz#84463226f7f36dc33592f6f4ed6374c48306c3f1"
+  integrity sha512-SbiLPU40JuJniHexQSAgad32hfwd+DRUdwF2PlVuI5RZD0/vahUco7R8vD86J/tcEKKF9vZrUVwgtmGCqlCKyA==
+
 randombytes@^2.0.0, randombytes@^2.0.1, randombytes@^2.0.5, randombytes@^2.1.0:
   version "2.1.0"
   resolved "https://registry.yarnpkg.com/randombytes/-/randombytes-2.1.0.tgz#df6f84372f0270dc65cdf6291349ab7a473d4f2a"
@@ -12119,6 +12157,11 @@
     mime-types "^2.1.27"
     schema-utils "^3.0.0"
 
+url-search-params-polyfill@^7.0.0:
+  version "7.0.1"
+  resolved "https://registry.yarnpkg.com/url-search-params-polyfill/-/url-search-params-polyfill-7.0.1.tgz#b900cd9a0d9d2ff757d500135256f2344879cbff"
+  integrity sha512-bAw7L2E+jn9XHG5P9zrPnHdO0yJub4U+yXJOdpcpkr7OBd9T8oll4lUos0iSGRcDvfZoLUKfx9a6aNmIhJ4+mQ==
+
 url@^0.11.0:
   version "0.11.0"
   resolved "https://registry.yarnpkg.com/url/-/url-0.11.0.tgz#3838e97cfc60521eb73c525a8e55bfdd9e2e28f1"

Run yarn install on the root of your GitLab instance and restart your GDK
Visit any dashboards such as http://gdk.test:3000/gitlab-org/gitlab-test/-/product_analytics/dashboards/dashboard_audience and validate that the widget appears with a chart being shown and the API request returns 201.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

I have evaluated the MR acceptance checklist for this MR.

Edited Nov 21, 2022 by Robert Hunt

Update the Cube proxy query param to Hash type to fix POST API calls

What does this MR do and why?

Screenshots or screen recordings

How to set up and validate locally

MR acceptance checklist

Merge request reports