Skip to content

Update file size validation

Carla Drago requested to merge 435485-update-size-validation into master

What does this MR do and why?

When downloading a note/comment attachment during GitHub import we should use an incremental check on the size of the file as it is downloaded in chunks to validate the size. Previously we were using the content-length from response headers to do this, but content-length is absent in the response headers returned from GitHub.

This also changes the file size limit to 100MB. This is in line with GitLab file size limits for comment attachments.

And it also removes the redundant validate_content_size check in the BulkImports::FileDownloadService.download_file method, as the method already checks the file size as the chunks are downloaded.

See #435485 (closed) See #429840 (closed)

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

  1. Create a project on GitHub
  2. Create an issue in the project
  3. Create a comment in the issue and attach a file
  4. Check out this branch and run the gdk against it
  5. Create a new Project by importing from GitHub
  6. Select the newly created GitHub project to import, ensuring that "Import issue and pull request events" and "Import markdown attachments (links)" are selected
  7. Import the project
  8. When the import is finished, visit the imported issue and observe the link to the comment attachment points to a gdk prefixed URL and not a GitHub one, e.g.
http://gdk.test:3000/root/new_pub_repo56/uploads/968ecbe1796caafc82f39c3e3d8586e0/5616349a-7cb7-40f6-95b2-46367a539d53

not

https://github.com/carlad/new_pub_repo/assets/1754975/5616349a-7cb7-40f6-95b2-46367a539d53

Related to #435485 (closed) and #429840 (closed)

Edited by Carla Drago

Merge request reports