Breaking changes to Runner because of steps-related work

Goal

This issue serves as a central record of breaking changes needed to support steps in the GitLab Runner. Whenever a new breaking change is discovered or introduced during development, it should be documented in the description below.

Exception process

Each breaking change needs to be approved in the breaking changes exception process. Create a product issue to start the process.

From the training:

If you can't share details about the current usage of a feature,
a request to remove it will very likely be denied. In addition to
having the count of users and frequency of feature use, you should
know the Tier and Platform of the impacted customers.

Issues should be submitted a minimum of six months in advance of the proposed removal timeframe.

Breaking changes

1. Job variables will no longer be set as environment variables on the build container

What changed:

• The build container now runs gitlab-runner-helper steps serve, which:
  • Starts the steps gRPC service
  • Listens on stdin for non-step script execution
  • This dual-purpose approach allows the same container to handle both steps and traditional scripts

Why this change was necessary:

• Previously, job variables were set as environment variables directly on the Docker build container
• While this worked for user scripts, it created a problem for steps: the step-runner could not distinguish between job variables and environment variables
• Removing variables from the container environment solves this ambiguity

Impact on existing functionality:

• Scripts: Not affected. Scripts are prepended with code that sets the necessary variables at runtime
• Container scanning: Customers who scan containers will see different results (no job variables in the environment)
  • This is actually an improved security posture

Reference: Introduced in step-runner#284 (closed)