Fix status cache for upstream pipelines

What does this MR do?

Related to #218964 (closed)

When the status of the downstream pipeline is updated, the upstream cache is not updated. This MR fixes this problem.

Why removed "cacheable?"

• As far as I understand from our EtagCaching middleware, we always create Etag for specific requests and we return the cached results until we expire them. (https://gitlab.com/gitlab-org/gitlab/-/blob/e152c806a772335433b4f7b95c000a9f142e4639/lib/gitlab/etag_caching/middleware.rb#L45-47)
• So, whether we call ExpirePipelineCacheWorker/ExpireJobCacheWorker or not, we always cache the paths for project_pipeline_path and project_build_path.
• Yet, we have an MR (!24443 (merged)) aiming to fix a cache problem by preventing caching by preventing calling ExpirePipelineCacheWorker. And I think that MR did not prevent caching because of the reason above.
• So I removed the cacheable? check for the ExpirePipelineCacheWorker.

Why removed EE of ExpirePipelineCacheService?

• I also used PipelineObjectHierarchy to touch every upstream and downstream pipeline.
• And moved that logic from EE to CE because parent-child pipelines are available for CE.

Why replaced with ExpirePipelineCacheWorker in ExpireJobCacheWorker?

• ExpirePipelineCacheWorker also expires other tags.

Screenshots (strongly suggested)

.gitlab-ci.yml

alpha:
  stage: deploy
  trigger:
    include: downstream.yml

beta:
  stage: deploy
  trigger:
    include: downstream.yml

downstream.yml

build:
  stage: build
  script:
    - sleep 7

test:
  stage: test
  script:
    - sleep 7

deploy:
  stage: deploy
  trigger:
    include: downdownstream.yml

downdownstream.yml

build:
  stage: build
  script:
    - sleep 7

test:
  stage: test
  script:
    - sleep 7

---

I did not use "Disable cache" when recording these:

Before

After

SQL Query

WITH RECURSIVE
"base_and_ancestors" AS (
  (SELECT "ci_pipelines".* FROM "ci_pipelines" WHERE "ci_pipelines"."id" = PIPELINE_ID)
  UNION
  (SELECT "ci_pipelines".* FROM "ci_pipelines", "base_and_ancestors", "ci_sources_pipelines"
                           WHERE "ci_sources_pipelines"."source_pipeline_id" = "ci_pipelines"."id"
                             AND "ci_sources_pipelines"."pipeline_id" = "base_and_ancestors"."id"
                             AND TRUE)
),
"base_and_descendants" AS (
  (SELECT "ci_pipelines".* FROM "ci_pipelines" WHERE "ci_pipelines"."id" = PIPELINE_ID)
  UNION
  (SELECT "ci_pipelines".* FROM "ci_pipelines", "base_and_descendants", "ci_sources_pipelines"
                           WHERE "ci_sources_pipelines"."pipeline_id" = "ci_pipelines"."id"
                             AND "ci_sources_pipelines"."source_pipeline_id" = "base_and_descendants"."id"
                             AND TRUE)
)

SELECT "ci_pipelines".*
FROM (
  (SELECT "ci_pipelines".* FROM "base_and_ancestors" AS "ci_pipelines")
  UNION
  (SELECT "ci_pipelines".* FROM "base_and_descendants" AS "ci_pipelines")
) ci_pipelines;

First run result: https://explain.depesz.com/s/eFBR

Second run result: https://explain.depesz.com/s/HgKC

Does this MR meet the acceptance criteria?

Conformity

• Changelog entry
• Documentation (if required)
• Code review guidelines
• Merge request performance guidelines
• Style guides
• Database guides
• Separation of EE specific content

Availability and Testing

• Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
• Tested in all supported browsers
• Informed Infrastructure department of a default or new setting change, if applicable per definition of done

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

• Label as security and @ mention @gitlab-com/gl-security/appsec
• The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
• Security reports checked/validated by a reviewer from the AppSec team