add API methods for binary packages to DL/UL multiple files
Problem to solve
I am currently trying to migrate CI and Artifact storage to gitlab, what I am missing is basically:
- uploading multiple files (compresses archives) manually
- downloading a whole package with CI
interaction with CI should be simple enough to be possible with shell scripts (so no complex json parsing/generating)
Intended users
User experience goal
Users should be able to upload a variable amount of files to a package, which will be including during a CI build. Examples would be secret keys, release specific documentation to be added to a binary file for redistribution.
In .gitlab-ci.yml
, a simple script should be able to
- list all available file(name)s, eg. a simple textformat with one filename per line
- download all files in a subdirectory
- upload multiple files
Proposal
Decide upon API functions to add
I believe the best way would be to add a new function for downloading an uncompressed .tar archive which contains:
- files in a subdirectory
#meta/
(first in file) containing all information necessary to fully recreate the repository, for example the json data from<api-projecturl>/packages/<package-id>
- all files with creation timestamp set correctly
- the tar should have a prefix, like
<package_name>-<package_version>/
this should allow easy importing andcat
-ing together multiple versions as later features
Expected use:
curl <API_URL> | tar x -C download_dir --strip-components=1 --exclude='#meta'
Reason would be that I assume those files would already be compressed due to limitations (no subdirectories, plain files will rather be stored in a git repo).
This fits all above requirements, including listing the files and would allow future improvements like restoring/importing packages from those archives. A separate function for just listing the files would be optional.
Similarly for uploading, a new function would accept such a archive. Expected use:
tar c -C upload_dir . | curl --upload-file - <API_URL>
(Note that the files will be prefixed with ./
)
Further details
As said, further possibilities would be:
- option to replace the package (instead of just adding/replacing files)
- export/import via Web UI
- more efficient way to list files in a separate API function
Permissions and Security
Documentation
Availability & Testing
What does success look like, and how can we measure that?
What is the type of buyer?
Is this a cross-stage feature?
Links / references
https://docs.gitlab.com/ee/user/packages/generic_packages/index.html