We noticed a significant discrepancy in the no. of `Migrations failed` in our [Sisense chart](https://app.periscopedata.com/app/gitlab/661967/Manage:Import-Dashboard?widget=15442655&udv=0) compared to the data in the DB as shown below: ### Sisense vs DB **Finished Migrations**: | Month | Sisense | DB| | ------ | ------ | ------ | | February | 2182 | 581 | | March | 2308 | 1379 | **Failed Migrations**: | Month | Sisense | DB| | ------ | ------ | ------ | | February | 482 | 240 | | March | 246 | 100 | ### Sisense SQL Tthe query for the [Sisense chart](https://app.periscopedata.com/app/gitlab/661967/Manage:Import-Dashboard?widget=15442655&udv=0) ```sql SELECT -- DATE_TRUNC(MONTH,ping_created_at) AS ping_created_at, ping_created_date_month, MAX(IFF(metrics_path = 'usage_activity_by_stage_monthly.manage.group_imports.gitlab_migration_failed', metric_value, NULL)) AS migration_failed, MAX(IFF(metrics_path = 'usage_activity_by_stage_monthly.manage.group_imports.gitlab_migration_finished', metric_value, NULL)) AS migration_finished, MAX(IFF(metrics_path = 'usage_activity_by_stage_monthly.manage.group_imports.gitlab_migration_timeout', metric_value, NULL)) AS migration_timeout FROM common_mart.mart_ping_instance_metric_monthly --this model will limit to the last ping of the month WHERE metrics_path IN ('usage_activity_by_stage_monthly.manage.group_imports.gitlab_migration_failed', 'usage_activity_by_stage_monthly.manage.group_imports.gitlab_migration_finished', 'usage_activity_by_stage_monthly.manage.group_imports.gitlab_migration_timeout') AND ping_delivery_type = 'SaaS' --this model is filtered to only include the SaaS production installation GROUP BY 1 ORDER BY 1 DESC ``` ### Data from Database #### February 2023 | Number of errors | Error message | | ------ | ------ | | 96 | Unsuccessful response 404 from [FILTER]. Body: {"message"=>"404 Not Found"} | | 46 | Net::ReadTimeout | | 36 | Visibility level internal has been restricted by your GitLab administrator | | 17 | unexpected character (after ) at line 1, column 1 [parse.c:733] in ' | | 10 | undefined method `version' for nil:NilClass | | 7 | Group URL has already been taken | | 7 | Unsuccessful response 502 from [FILTER] | | 6 | Name can contain only letters, digits, emojis, '_', '.', dash, space, parenthesis. It must start with letter, digit, emoji or '_'. | | 5 | Unsuccessful response 403 from [FILTER]. Body: {"message"=>"403 Forbidden"} | | 4 | Unsuccessful response 404 from [FILTER]. Body: {"message"=>"404 Group Not Found"} | | 3 | Require two factor authentication is forbidden by a top-level group | | 2 | User not allowed to create group | | 2 | Unsuccessful response 500 from [FILTER] | | 1 | Unsuccessful response 401 from [FILTER]. Body: {"error"=>"invalid_token", "error_description"=>"Token was revoked. You have to re-authorize from [FILTER]."} | | 1 | Unsuccessful response 503 from [FILTER] | #### March 2023 | Number of errors | Error message | | ------ | ------ | | 57 | Unsuccessful response 404 from [FILTER]. Body: {"message"=>"404 Not Found"} | | 17 | Visibility level internal has been restricted by your GitLab administrator | | 8 | Net::ReadTimeout | | 6 | User not allowed to create group | | 4 | Require two factor authentication is forbidden by a top-level group | | 3 | Unsuccessful response 404 from [FILTER]. Body: {"error"=>"404 Not Found"} | | 2 | Group URL has already been taken | | 1 | Unsuccessful response 401 from [FILTER]. Body: {"error"=>"invalid_token", "error_description"=>"Token was revoked. You have to re-authorize from [FILTER]."} | | 1 | Unsuccessful response 403 from [FILTER]. Body: {"message"=>"You cannot perform write operations on a read-only instance"} | | 1 | Unsuccessful response 403 from [FILTER]. Body: {"message"=>"403 Forbidden"} |