Skip to content

First run for Snowplow HAML event dictionary entries

What does this MR do and why?

Related to #333992 (closed)

This covers all Snowplow events emitted from frontend using data-attributes (https://docs.gitlab.com/ee/development/snowplow/implementation.html#data-track-attributes) until date. These are mostly used on HAML templates.

As this is a first pass, it's acceptable that some non-required data are missing or invalid. Any missing event will be added on a follow-up pass.

This is a significant MR, but the changes are easy to comb through. And these definitions are taking the improvements at !73794 (merged) into account.

Generator used
/* eslint-disable import/no-commonjs */
/* eslint-disable no-use-before-define */
/* eslint-disable no-restricted-syntax */
const fs = require('fs');
const csv = require('csv-parser');

const SOURCE_FILE_PATH = './events.csv';
const FOLDER_PATH = './event_dictionary_generator_output';
const ARRAY_SEPARATOR = ';';

// extra_properties
const baseTemplate = `description:
category:
action:
label_description:
property_description:
value_description:
extra_properties:
identifiers:
product_section: ""
product_stage:
product_group:
product_category:
milestone:
introduced_by_url:
distributions:
tiers:
`;

createFolderIfNeeded();
iterateRows();

function createFolderIfNeeded() {
  if (!fs.existsSync(FOLDER_PATH)) {
    fs.mkdirSync(FOLDER_PATH);
  }
}

function iterateRows() {
  fs.createReadStream(SOURCE_FILE_PATH).pipe(csv()).on('data', createFileFromRow);
}

function createFileFromRow(row) {
  let template = String(baseTemplate);

  for (const [name, rawValue] of Object.entries(row)) {
    const label = `${name}:`;
    let value = rawValue;
    const isArray = value.includes(ARRAY_SEPARATOR) || ['distributions', 'tiers'].includes(name);

    if (name === 'category') {
      value = value || 'default';
    }

    if (['description', 'milestone', 'label_description', 'property_description', 'value_description'].includes(name)) {
      value = `"${value}"`
    }

    if (isArray) {
      const items = value.split(ARRAY_SEPARATOR).join('\n- ');
      template = template.replace(label, `${label}\n- ${items}`);
    } else {
      const variableValue = value.replace(/^`(.+?)`$/, '"`$1`"');
      template = template
        .replace(label, `${label} ${variableValue}`);
    }
  }

  // Timestamp
  const pad = (number, at = 2) => `0${number}`.slice(at * -1);
  const now = new Date;
  const timestamp = [
    now.getFullYear(),
    pad(now.getMonth() + 1),
    pad(now.getDate()),
    pad(now.getHours()),
    pad(now.getMinutes()),
    pad(now.getSeconds()),
  ].join('');
  // Custom slug logic attempt
  const filename = `${timestamp}_${row.category || 'default'}_${row.label_description}_${row.action}`
    .replace(/[`\[\]]/g, '')
    .replace(/(?: \| |:)/g, '_')
    .replace(/\s+/g, '_')
    .toLowerCase()
    .trim()
    .replace(/_{2,}/, '_');

  const path = `${FOLDER_PATH}/${filename}.yml`;
  // Create file
  fs.writeFileSync(path, template);
  console.info(path, 'created');
}

Screenshots or screen recordings

These are strongly recommended to assist reviewers and reduce the time to merge your change.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Axel García

Merge request reports