Include keep-around and other references in the checksum calculation

See https://gitlab.com/gitlab-org/gitlab-ee/issues/5196