Async MR Pipeline creation - iteration 1

Problem

In !161407 (merged) we introduced the ability to trigger pipelines for an MR async behind a feature flag. Before rolling out this feature flag to customers we likely want to improve the user feedback so that they can see a spinner while their pipeline is still being created and so that they can see if there were any validation errors.

Since the pipeline creation is async this is much trickier to implement than before because we need to actually persist some data even if the pipeline is not created. And we need a new dedicated API for streaming updates for that data with a special ID the user can use to track it.

Proposed Solutions

We are currently exploring a Frontend only solution and will update this issue with the resolution.

Return a unique ID from this API and pass that ID to the worker. When the worker finishes update some data in Redis with this ID (and a short expiry like ~5 minutes). The user can poll an API with the ID to see the state. This will allow us to implement basically the same UX we have for sync pipeline creation with the same spinner and flash errors.

We could even consider making this part of the existing polling that is going on on this page. We could either update the polling to pass in the ID of last async creation or maybe even simpler we can just associate all async creations with a given user. And whenever they are polling they'll get back a status on the latest async creations for that MR. Or it could be added to the user session that they are waiting on that async creation. If added to the session this can all be done in the backend and the frontend doesn't even need to keep track of the id and the result will just come back in the polling response.

Context

The following discussion from !161407 (merged) should be addressed:

• @allison.browne started a discussion: (+2 comments)

  Not returning a validation error can create a UX change.

  We return validation errors sometimes which appear to be rendered by the client as a generic alert message:

  https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/ci/merge_requests/components/pipelines_table_wrapper.vue#L204

  For conditions like rate limiting we don't persist an error on the pipeline because we don't want to run all of the associated workers with spinning a pipeline down, which could cause incidents if the user is creating many pipelines: https://gitlab.com/gitlab-org/gitlab/-/blob/ea46197988ec32ffc4d8725ec1d89259f2671d31/spec/services/ci/create_pipeline_service/rate_limit_spec.rb#L35

  In that condition we have a validation error on the pipeline so it should display an alert when the pipeline creation request ends.

  I think we start to complicate the MR a bit if we preserve this behavior because we need a way to keep track of pipelines that were not persisted and a way to poll about them. What do you think?

Close Criteria

• We have the ff completely rolled out
• We have the skeleton spinner (task Briley is assigned to)