Not pulling docker images often enough (or at all?)

Seems like the runner will pull the image once when a job is first run, but then afterwards the images are never updated:

gitlab@gitlab-ci-runner:~$ docker images
REPOSITORY                                             TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
gitlab/gitlab-runner                                   cache               4f1f3bc1bfbf        2 weeks ago         2.433 MB
python                                                 2                   65c00af7c3f9        8 weeks ago         746.6 MB
busybox                                                latest              8c2e06607696        3 months ago        2.433 MB
ubuntu                                                 latest              2d24f826cb16        5 months ago        188.3 MB

Notice that both python:2 and ubuntu:latest have very old images, and there are newer ones available for both.