Caching general build artifacts between stages

I'm not sure why it's not working, maybe I did understand the use case wrong for cache.
What I wanted to do:
Define 2 stages => build and deploy
In the build stage our app gets put together and into a new folder ./build/
This folder needs to be carried over to the deploy stage so it can be deployed on a server.

Here's what the .gitlab.yml file looks like (shortened)

image: ctsmedia/gitlab-runner-build-base-cts:latest
stages:
  - build
  - deploy

before_script:
  - export DEBIAN_FRONTEND=noninteractive
...

build:
  stage: build
  artifacts:
    paths:
      - build/
  cache:
    paths:
      - build/
  script:
    - phing purge
    - sh bin/ci-merge-feature-branches.sh
    - phing build

dev:
  stage: deploy
  script:
    - apt-get -qqy install rsync
    - rsync -ae ssh ./build/* user@IP:/super/secret/path
  only:
      - /^issue-.*$/
      - /^iss#.*$/
      - /^feature.*$/
      - develop

The build run succeeds but the build folder gets deleted when switched to the deploy stage:

gitlab-ci-multi-runner 0.7.2 (998cf5d)
Using Docker executor with image ctsmedia/gitlab-runner-build-base-cts:latest ...
...
Running on runner-5dfeb163-project-2-concurrent-0 via 1112eb26ef3f...
Fetching changes...
Removing artifacts.upload.log
Removing build/
Removing node_modules/
HEAD is now at 2d7994e Merge remote-tracking branch 'refs/remotes/origin/issue-5' into develop
Checking out ceae231a as develop...

For now I just use one stage instead of which is perfectly fine and has less overhead because there is only run to be made. The build folder and all it's content is not tracked and gets created during the build step.