Skip to content

Add script to migrate InfluxDB e2e test data to GCS in json format

What does this MR do and why?

Closes gitlab-org/quality/quality-engineering/team-tasks#2703 (closed)

This MR:

  1. Converts MigrateInfluxDataToGcs to a super class and moves CSV specific functionality to MigrateInfluxDataToGcsCsv
  2. Adds MigrateInfluxDataToGcsJson that creates json objects and uploads to a GCP bucket.

Please note that there are no code changes in MigrateInfluxDataToGcsCsv. It has the same code moved over from MigrateInfluxDataToGcs.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

  1. Download and save the json file in 1P attached to a note titled "influx-to-gcs GCS service account in gitlab-qa-resources". The path to the saved file should be provided in QA_METRICS_GCS_CREDS in the below command.

  2. From the qa folder, run:

QA_INFLUXDB_TIMEOUT=300 QA_METRICS_GCS_PROJECT_ID="gitlab-qa-resources" QA_METRICS_GCS_CREDS="/path/to/json/key/file.json" QA_METRICS_GCS_BUCKET_NAME="e2e_test_metrics_json" QA_INFLUXDB_TOKEN="<QA_INFLUXDB_TOKEN_FROM_1P>" QA_INFLUXDB_URL="https://influxdb.quality.gitlab.net" bundle exec rake "influx_to_gcs_json[2024, 1, 1]"
Edited by Sanad Liaquat

Merge request reports