Fix job depencies view in the pipeline graph

What does this MR do and why?

When creating the new job dependencies view, we would not take into account stage jobs (meaning jobs without needs) and they would all land in the first column. This is inaccurate as stage job depends on all the jobs from the previous stage. So visually, we would get only job with needs as new columns. This MR fixes this and now stage jobs are also considered with for new columns. More technical details below.

Technical details

Use the new previousStageJobsOrNeeds value from the API which returns: The needs if these values are specified inside the CI configuration OR The jobs from the previous stage if the job has no needs keywords specified.

From there, we can adjust the algorithm that build the columns in the job dependencies view. Part of the solution fixes itself by just changing which array of needs we pass through: the original needs or the previousStageJobsOrNeeds. By passing the execution requirements, we can leverage D3 createSankey to determine "layers" which is based off how many jobs are required by the job it itself requires. That way we can guarantee which column the job should be in.

The other adjustment is that we don't want lines drawn from a job to all the job from a previous stage (which would be a performance nightmare AND visually impossible to distinguish the lines) so what we do is that we run the algorithm twice: once for the column and once for the lines. For line, we use the needs data and a job with no explicit needs will have no lines drawn.

There was no need to add tests here because we would be testing D3 implementation of Sankey. We had thought of visual testing in the past, but this is still not an option for now. Also, we cannot test this in integration tests or even E2E because all we are changing is how the lines are drawn: there is no interaction to test.

Screenshots or screen recordings

Stage view	Before	After

How to set up and validate locally

1.Navigate to CI/CD => Editor 2.Use a yaml configuration with needs. For example:

stages:          # List of stages for jobs, and their order of execution
  - build
  - test
  - deploy
  - new

build-job:       # This job runs in the build stage, which runs first.
  stage: build
  script:
    - echo "Compiling the code..."
    - echo "Compile complete."

unit-test-job:   # This job runs in the test stage.
  stage: test    # It only starts when the job in the build stage completes successfully.
  script:
    - echo "Running unit tests... This will take about 60 seconds."
    - sleep 60
    - echo "Code coverage is 90%"

lint-test-job:   # This job also runs in the test stage.
  stage: test    # It can run at the same time as unit-test-job (in parallel).
  script:
    - echo "Linting code... This will take about 10 seconds."
    - sleep 10
    - echo "No lint issues found."

deploy-job:      # This job runs in the deploy stage.
  stage: deploy  # It only runs when *both* jobs in the test stage complete successfully.
  script:
    - echo "Deploying application..."
    - echo "Application successfully deployed."
  needs: ['build-job']

execute-first:
  stage: deploy
  needs: []
  script: echo now

3. Run a pipeline (you don't need a working runner, the pipeline will get created and then stay in pending)
4. Go in CI/CD => Pipelines
5. Use the view selector above the graph to see "Job Depencies"
6. Notice that jobs are in the proper column given their execution order
7. To verify this: A job with explicit needs will be shown to the right of its requirement, a job with an empty array of needs will always be shown in the first column and a job with NO needs will be shown in a "stage" order, meaning always one column to the right of the previous stage jobs

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.

Related to #340057 (closed)