Remove certifi dependency that violates GitLab license policy

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Problem

The certifi package, an indirect dependency for DAST API/API Fuzzing, is licensed under the MPL. This license is not approved by GitLab legal. An exception has been granted, but the license policy feature has some bugs that on occasion prevent MRs from being merged and ask for an approval from legal. This in turn impacts the speed of development.

certifi is a convenience package that can be removed as a dependency if urllib3 is upgraded to a recent version. Upgrading urllib3 will require building it from source as our OS distributions are still using an older version.

The certifi package contains a curated list of root TLS certificates from Mozilla. The OS distribution also includes a bundle of root certificates that can be used instead.

Implementation plan

  1. Verify that upgrading urllib3 to v2 is compatible with other dependencies such as requests.
  2. Build urllib3 from source and install during our base image builds. See the history of our build jobs for an example of building cryptography from source. Avoid adding build-tools to our final image.
  3. Update Python projects such as worker-entry and SDK/libraries/python to use our built urllib3 and remove references to certifi.
  4. Verify all tests are passing.
  5. Manually verify worker-entry can validate https://gitlab.com
Edited by 🤖 GitLab Bot 🤖