Docs feedback: PyPI API package upload does not mention file digests (in particular, the SHA-256 checksum)

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Work on this issue
• Close this issue

At https://docs.gitlab.com/ee/api/packages/pypi.html#upload-a-package the PyPI API docs explain how to send a multipart form via POST request to upload a package file to the Python package registry in a GitLab project.

However, the documentation makes no mention of the file digests that should also be provided as form parameters; the upload will succeed as documented, but the file_sha256 field in the uploaded package file's metadata will be null.

This will subsequently prevent a successful download of the uploaded package file via the same API; as documented at https://docs.gitlab.com/ee/api/packages/pypi.html#download-a-package-file-from-a-project, the SHA-256 checksum is required to access the uploaded file, but without that metadata field, the download will fail with an error.

The PyPI API does provide additional form parameters, in particular sha256_digest for the POST request when uploading a package file; if this is provided with the file's SHA-256 checksum, then the uploaded package file's metadata will contain the required value for the file_sha256 key, and subsequently downloading the package file via the PyPI API will work as expected.

I suggest adding the missing details about the sha256_digest form parameter (and possibly also other supported digests, such as md5_digest) to the documentation at https://docs.gitlab.com/ee/api/packages/pypi.html#upload-a-package.

⚙️ Technical considerations

• PyPI uploads are workhorse assisted uploads. As such, workhorse will compute several digest during the upload and passe them to rails when confirming the upload.
• Currently, rails set the file_x digests columns from the parameters sent by package manager clients (x_digest parameters). However, from this issue description, it seems that sometimes, these are not correctly sent and so rails will set some file_x columns to NULL. This in turn makes the package unpullable as the pull operation is based on the sha256 digest.
• To fix this:
  • Read the digests from workhorse.
  • If the package manager client sent some digests, compare them with the ones from workhorse. If there is a mismatch, reject the upload (400 Bad Request).
  • Set the file_x digests columns with the digests from workhorse.
  • Apply the above changes with a feature flag as we could be breaking the upload feature in the PyPI support.