WIP: Implement runnershell for predefined stages

What does this MR do?

Adds an integrated runnershell for executing predefined stages.

runnershell wraps an existing shell implementation, but takes over for predefined stages. Our abstract shell is still used to generate the "script" that runnershell ultimately executes.

The script is a json encoded list of actions to be executed.

---

The Docker and Kubernetes executors already have a sort of split execution. "Predefined" stages go to the helper container, and user-script stages go to the main build container.

This MR introduces this split of execution for other executors also. Executing either runnershell script, or the user's chosen shell (powershell, bash) depending on the stage.

---

Some user-scripts are also executed during predefined stages, such as pre_clone_script. runnershell executes these in a subshell, by opening a new process up based on the wrapped shell's commands. The script is then written to stdin to be executed.

There's some differences in behaviour here though:

• Because it's a subshell, any exported environment variables in pre_clone_script will not be available when running the get_sources task.
• The underlying shell has to support stdin execution. Which, is, almost all of them anyway. frowns at cmd.

---

This MR does NOT re-implement artifacts-uploader, cache-uploader, the get_sources logic in Go.

Instead, it relies on the Abstract Shell, and builds a list of commands to be executed, in the same way as any other supported shell.

Due to this, it only has to support a limited range of functionality that abstract shell requires:

Supporting: Line, Cd, Mkdir, Rm, Print, Cmd and IfDirectory, IfFile, IfCmd, Else.

Technically, we could remove IfDirectory and IfFile, as these are not used by the abstract shell.

Playing with the shell

# run a command from the shell (git version) with output
echo '{"actions":[{ "cmd": {"args": ["git", "--version"], "output": true} }]}' | ./gitlab-runner runnershell

# run a 'line' of script, using the inner shell definition
echo '{"shell-command":["bash"], "actions":[{"line":"echo hello"}, {"line":"echo hi"}]}' | ./gitlab-runner runnershell

# if a directory exists or doesn't exist, write a message (also enable tracing)
echo '{
  "trace": true,
  "actions":[
    {
      "if": {
         "directory": ".",
         "actions": [{ "print": "directory exists" }],
         "else": [{ "print": "directory does not exist" }]
      }
    }
  ]
}' | ./gitlab-runner runnershell

Why was this MR needed?

To eventually reduce the complexity of adding new shells. If a shell only has to handle user-defined scripts, there's no need to create a large abstraction for each one. A new shell could perhaps even be a user defined configuration, without any code needing to be written.

It's also much faster than invoking powershell, so a runnershell wrapped-powershell shell (say it quickly), should execute faster than powershell alone.

What's the best way to test this MR?

🤞

What are the relevant issue numbers?

#1687