Parallel Execution and Other Strategies
A.k.a. "step one: make a pizza"
The in-built steps
type defines a linear sequence of sub-steps, each step (maybe) referencing the outputs of the previous. Additional executions strategies are possible, such as parallel execution in which all steps are executed concurrently and no step can reference another.
There may be a setting to control parallelism, or to group steps into "stages". Maybe to run each step in a separate VM or on a separate machine entirely. Steps execution strategies begin to approach the fine-grained controls of a workflow. This is okay. In fact the workflow is itself a single step with an elaborate execution strategy.
E.g. the process of making a pizza can be divided in "stages" such as "make the dough", "chop the ingredients", "dress the pizza", "bake the pizza". Each state itself can be divided into sub- stages such as "fetch the ingredients", "fetch a knife", "chop the ingredients", "place into bowls". Likewise the overall process can be described as a single step "make a pizza".
A unified model of workflow and steps allows growth and adaptation with low impedance mismatch. Methods of fine-grained control and abstraction are available on any level where they are needed, now or in the future. After all, "make a pizza" may become embedded as a sub-step in "feed the kids".
Delegation of Strategy
The robust and opinionated selection of controls available in a GitLab workflow do not lend themselves to a lean core. Nor are they necessarily compatible with other, equally valid sets of controls such as "workflows" for other problem domains. This presents a problem for step-runner which relys on a stable and universal core set of functionality. We don't want to build "stages" etc. into step-runner.
The approach adopted by step-runner for such situations is to delegate control to a step which specializes in a particular method of execution. For example, running steps in a container is delegated to a single step which can fetch and initialize the desired container. Control is then delegated to another step-runner inside and the output reified into step results upon completion.
Any strategy beyond linear execution should be delegated to a step. For example "parallel" execution will be handled by a parallel step which makes gRPC calls to (or execs) step-runner for each sub-step. That means without containerization, a single instance of step-runner may be executing both top-level steps (jobs) and sub-steps. It doesn't matter because the handling is the same. Only care must be given to prevent run request id collisions.
So a strategy which implements the controls of a GitLab workflow would be a single, specialized step. Special keywords and syntactic sugar are either compiled into additional steps (such as the handling of the keyword "script") or they are delegated one-by-one back to step-runner as needed by the "workflow" step implementation, which can be written in any language. E.g. "stages" of steps will be given back to step-runner by making a gRPC call to the step-runner service local socket.