Skip to content

Parse all failed files in pipeline

Alex Pooley requested to merge 434098-parse-all-failed-files-in-pipeline into master

What does this MR do and why?

This is a script to parse a pipeline and dump all rspec files that had failures. This script is specifically needed by !137713 (merged) and follow up processes to remove the shims.

How to set up and validate locally

  1. Setup a personal access token with read_api access at https://gitlab.com/-/profile/personal_access_tokens
  2. Call the script that references a pipeline with failed jobs. E.g. tooling/bin/failed_spec_files -t <API token> -p 278964 -l 1092874549 -o /tmp/out.yml -d

Output will match:

Fetching failed jobs... found 22
Fetching job logs for #5665879558
Fetching job logs for #5665879527
  ee/spec/requests/api/graphql/mutations/security_policy/commit_scan_execution_policy_spec.rb
Fetching job logs for #5665879526
  ee/spec/requests/api/graphql/project/pipelines/dast_profile_spec.rb
  ee/spec/requests/api/project_milestones_spec.rb
Fetching job logs for #5665879523
  ee/spec/controllers/ee/profiles/preferences_controller_spec.rb
  ee/spec/requests/trial_registrations_controller_spec.rb
Fetching job logs for #5665879521
  ee/spec/controllers/profiles/billings_controller_spec.rb
  ee/spec/requests/projects/issues_controller_spec.rb
  ee/spec/requests/users_controller_spec.rb
Fetching job logs for #5665879520
  ee/spec/controllers/projects/iterations_controller_spec.rb
  ee/spec/requests/api/graphql/mutations/security_policy/unassign_security_policy_project_spec.rb
  ee/spec/requests/projects/analytics/code_reviews_controller_spec.rb
Fetching job logs for #5665879519
  ee/spec/requests/api/graphql/project/path_locks_spec.rb
  ee/spec/requests/api/search_spec.rb
Fetching job logs for #5665879518
  ee/spec/requests/api/graphql/project/path_locks_spec.rb
  ee/spec/requests/trial_registrations_controller_spec.rb
Fetching job logs for #5665879516
  ee/spec/controllers/ee/profiles/preferences_controller_spec.rb
  ee/spec/controllers/profiles/billings_controller_spec.rb
  ee/spec/controllers/projects/iterations_controller_spec.rb
  ee/spec/requests/api/graphql/project/pipelines/dast_profile_spec.rb
  ee/spec/requests/projects/analytics/code_reviews_controller_spec.rb
Fetching job logs for #5665879513
  ee/spec/requests/api/graphql/mutations/security_policy/unassign_security_policy_project_spec.rb
Fetching job logs for #5665879512
  ee/spec/requests/api/graphql/mutations/security_policy/commit_scan_execution_policy_spec.rb
  ee/spec/requests/projects/issues_controller_spec.rb
Fetching job logs for #5665879511
  ee/spec/requests/api/project_milestones_spec.rb
  ee/spec/requests/api/search_spec.rb
  ee/spec/requests/users_controller_spec.rb
Fetching job logs for #5665879361
  spec/lib/gitlab/error_tracking/processor/sidekiq_processor_spec.rb
Fetching job logs for #5665879256
Fetching job logs for #5665878939
  spec/lib/gitlab/error_tracking/processor/sidekiq_processor_spec.rb
Fetching job logs for #5665878618
Fetching job logs for #5665878614
Fetching job logs for #5665878128
  spec/frontend/fixtures/issues.rb
  spec/frontend/fixtures/namespaces.rb
  spec/frontend/fixtures/pipeline_details.rb
  spec/frontend/fixtures/snippet.rb
  spec/frontend/fixtures/users.rb
Fetching job logs for #5665878122
  spec/frontend/fixtures/namespaces.rb
Fetching job logs for #5665878119
  ee/spec/frontend/fixtures/search.rb
  spec/frontend/fixtures/issues.rb
Fetching job logs for #5665878114
  spec/frontend/fixtures/webauthn.rb
Fetching job logs for #5665878062
  spec/frontend/fixtures/snippet.rb
  spec/frontend/fixtures/users.rb
Wrote 22 files to /tmp/out.yml

/tmp/out.yml has a YAML array of files that failed in the pipeline.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #434098 (closed)

Edited by Alex Pooley

Merge request reports