Artifacts browser caches metadata.gz from object storage, but the cache data won't be deleted
Summary
If job artifacts and metadata are uploaded to ObjectStorage, GitLab-rails tries to cache the file to local storage(shared/artifacts/tmp
). This cached file is to be removed, however it's not been removed, currently.
Steps to reproduce
- Configure GDK with minio(ObjectStorage)
- Run a pipeline with
artifacts
keyword in gitlab-ci.yml - Visit artifacts browser. e.g. project_id/-/jobs/job_id/artifacts/browse/data/
- Observe Carrierwave cache dir. e.g.
/Users/hoge/Documents/gitlab/gdk-ee/gitlab/shared/artifacts/tmp/cache/1517908643-14076-0001-5842
What is the current bug behavior?
- It's not deleted
What is the expected correct behavior?
- It's deleted after
use_file
Possible fixes
Currently, we're removing with cache_storage.delete_dir!(cache_path(nil))
https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/app/uploaders/object_storage.rb#L144
def use_file
if file_storage?
return yield path
end
begin
cache_stored_file!
yield cache_path
ensure
cache_storage.delete_dir!(cache_path(nil))
end
end
However, this can't delete because the directory isn't empty
https://github.com/carrierwaveuploader/carrierwave/blob/master/lib/carrierwave/storage/file.rb#L102
##
# Deletes a cache dir
#
def delete_dir!(path)
if path
begin
Dir.rmdir(::File.expand_path(path, uploader.root))
rescue Errno::ENOENT
# Ignore: path does not exist
rescue Errno::ENOTDIR
# Ignore: path is not a dir
rescue Errno::ENOTEMPTY, Errno::EEXIST
# Ignore: dir is not empty
end
end
end
We should consider using FileUtils.rm_r
.