ssh-agent not available if started from script

Summary

In my job I have a script that needs to be executed on a remote machine, so I use SSH to connect to it. I followed this article with some minor changes to make it work on alpine.

It all works as long as all the commands are in .gitlab-ci.yml file line by line, but if I put the SSH initialization into a separate script file, ssh-agent and the keys added are no longer available once outside the script.

DISCLAIMER: This might be entirely my misunderstanding of how things work on linux.

Steps to reproduce

So here are my files. This is all based on https://gitlab.com/gitlab-examples/ssh-private-key.

.gitlab-ci.yml:

image: alpine:latest

before_script:
- chmod +x before_script.sh
- ./before_script.sh
- echo "list keys (outside script, before_script)"
- ssh-add -l

Test SSH:
  script:
  - echo "list keys (outside script, script)"
  - ssh-add -l

  # try to connect to GitLab.com
  - ssh git@gitlab.com

  # try to clone yourself, the SSH_PRIVATE_KEY was added as deploy key to this repository
  - git clone git@gitlab.com:gitlab-examples/ssh-private-key.git

before_script.sh:

#!/bin/sh

# install ssh-agent (and git, as alpine doesn't come with it)
echo "add"
apk add --update openssh-client git

# run ssh-agent
echo "run"
eval $(ssh-agent -s)

# add ssh key stored in SSH_PRIVATE_KEY variable to the agent store
echo "ssh-add"
echo "$SSH_PRIVATE_KEY" > temp-key
chmod 600 temp-key
ssh-add temp-key
rm temp-key

# disable host key checking (NOTE: makes you susceptible to man-in-the-middle attacks)
# WARNING: use only in docker container, if you use it with shell you will overwrite your user's ssh config
echo "config"
mkdir -p ~/.ssh
echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config

echo "list keys (inside script)"
ssh-add -l

There are some extra echos in there for debugging purposes.

Actual behavior

Here's the log from the CI:

Running with gitlab-ci-multi-runner 9.1.0 (0118d89)
  on docker-auto-scale (e11ae361)
Using Docker executor with image alpine:latest ...
Using docker image sha256:69fe920d20f8e2f59c63f74c3bff548c7907956b4f6e576cc1d7c166ac36e10a for predefined container...
Pulling docker image alpine:latest ...
Using docker image alpine:latest ID=sha256:4a415e3663882fbc554ee830889c68a33b3585503892cc718a4698e91ef2a526 for build container...
Running on runner-e11ae361-project-3212951-concurrent-0 via runner-e11ae361-machine-1493539866-a69e5396-digital-ocean-2gb...
Cloning repository...
Cloning into '/builds/gligoran/ssh-private-key'...
Checking out 369b51c6 as master...
Skipping Git submodules setup
$ chmod +x before_script.sh
$ ./before_script.sh
add
fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/community/x86_64/APKINDEX.tar.gz
(1/7) Installing ca-certificates (20161130-r1)
(2/7) Installing libssh2 (1.7.0-r2)
(3/7) Installing libcurl (7.52.1-r3)
(4/7) Installing expat (2.2.0-r0)
(5/7) Installing pcre (8.39-r0)
(6/7) Installing git (2.11.1-r0)
(7/7) Installing openssh-client (7.4_p1-r0)
Executing busybox-1.25.1-r0.trigger
Executing ca-certificates-20161130-r1.trigger
OK: 26 MiB in 18 packages
run
Agent pid 16
ssh-add
Identity added: temp-key (temp-key)
config
list keys (inside script)
4096 SHA256:MziaFQCAWxx5m/11c+dt/hDyzJShQwgArOZWI+NiGI8 temp-key (RSA)
$ echo "list keys (outside script, before_script)"
list keys (outside script, before_script)
$ ssh-add -l
Could not open a connection to your authentication agent.
ERROR: Job failed: exit code 2

As you can is in the last few lines, when listing the keys in the ssh-agent, they're available when still inside before_script.sh, but once you get out of it, they and also the ssh-agent are not available anymore.

Expected behavior

I would expect to see all ssh-add -l list the key added in the script and the ssh-agent to still be available outside the script and in the script: part of the job.

You can also find my code here: https://gitlab.com/gligoran/ssh-private-key (it should be public).