Browse build artifact files with the GitLab UI

Tasks:

  • Prepare database

  • Create a new table (ci_build_arifacts) (not now)

  • Migrate artifacts_file from ci_build to new table (not now)

  • Update codebase to use Ci::Build.artifacts (not now)

  • Add expiration date to artifacts (#3439 (closed)) (not now)

  • Add artifacts metadata field to build

  • Prepare fixtures

  • Ci Build fixture

  • Artifacts archive fixture

  • Artifacts metadata fixture

  • Prepare mechanism that will generate metadata

  • Decide if we prefer to generate metadata by runner or after receiving artifacts archive

  • Mock metadata at the beginning (fixtures, seeds)

  • Implement mechanism that will generate metadata

  • Change compression format used by CI Runner if decided on other than tar.gz

  • Design mechanism that will mitigate possible DoS attack when malicious artifacts archive is received

  • Implement DoS mitigation mechanism in Workhorse

  • Artifacts contents view

  • Design mechanism that will make iterating metadata collection easier

  • Implement mechanism that will make iterating metadata collection easier

  • Create a new view, where content of artifacts will be displayed

  • Make ground for multiple artifacts when designing this view (not now)

  • Consult Frontend / UX

  • Tweak performance of metadata's paths traversal

  • Check for possible vulnerabilities in handling arbitrary paths in metadata

  • Choose compression format

  • Prepare benchmarks for ZIP / TAR GZIP / TAR XZ

  • Check ZIP

  • Check how ZIP behaves with big files (> 4G)

  • Check how ZIP handles absolute paths

  • Check how ZIP handles UIDs/GIDs, UNIX permissions

  • Check how ZIP handles SUID/SGID

  • Check how ZIP handle case sensitivity

  • Check for security vulnerabilities in ZIP (browse CVEs)

  • Protoype / benchmark ZIP implementation in Go

  • Extracting single file

  • Making a list of files

  • Compare to CLI ZIP implementation

  • Check large file support (> 4G)

  • Check Ruby ZIP implementations

  • Check ruby-zip gem

  • Serve artifacts metadata through API

  • Add endpoints for artifacts metadata

  • Check if this can be used in hooks

  • Make it possible to download some files from artifacts archive via GitLab UI

  • Add implementation for creating TAR/ZIP metadata

  • Prepare mechanisms that will extract single file from artifacts archive. This should be done by workhorse.

  • Design caching mechanism for extracted files

  • Modify view, add splashscreen/modal

  • Consult Frontend / UX

  • Cleanup

  • Invalidate artifacts cache

  • Prepare a cleanup

  • Remove expired artifacts (#3439 (closed)) (not now)

  • Documentation and dependencies

  • Write documentation for artifacts browser

  • Add Info-ZIP (unzip) dependency to docker images, Omnibus, etc. (we do not want to add another external dependency)