Enable and remove ci_optional_needs_for_cross_pipeline FF

Furkan Ayhanrequested to merge
remove-ci-optional-needs-for-cross-pipeline-ff into master

What does this MR do and why?

Removes the ci_optional_needs_for_cross_pipeline feature flag. The flag is fully enabled on GitLab.com.

With this change, optional: true with needs:pipeline:job is always supported. This MR also removes the legacy code path for cross-pipeline dependencies.

Related to #630677 (closed)

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist.

Merge request reports