Skip to content

Include repo archiving via UI and API in projects download throttling

Eugie Limpin requested to merge el-rate-limit-repo-download-on-archive into master

Resolves https://gitlab.com/gitlab-org/modelops/anti-abuse/team-tasks/-/issues/65

What does this MR do and why?

Previously, users are banned (from the application or a namespace) only when they exceed the threshold for project downloads when cloning via SSH and HTTP. This MR updates the feature to apply the same restriction when users download repositories via UI and API (archive).

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

How to set up and validate locally

Validate user is banned from the application when exceeding projects download threshold

  1. Ensure you are running GDK with an Ultimate license. The following command should log true if this is set up correctly

    echo "License.feature_available?(:git_abuse_rate_limit)" | rails c

  2. Turn on the following feature flags:

    $ rails console
> Feature.enable(:git_abuse_rate_limit_feature_flag)
> Feature.enable(:auto_ban_user_on_excessive_projects_download)

  3. Using an admin user (root), set the application settings for the feature

    $ rails console
> ApplicationSetting.first.update({
   max_number_of_repository_downloads: 1,
   max_number_of_repository_downloads_within_time_period: 300,
   auto_ban_user_on_excessive_projects_download: true
})

  4. Create a private top-level group and two projects under the group

  5. Add a user as a developer/maintainer to the group

  6. Using the developer/maintainer user from the previous step, download an archive of the first project

    📸 Screen_Shot_2022-10-20_at_1.58.25_PM

  7. Validate that the download works as expected

  8. Download an archive of the second project

  9. Validate that the download does not work (user is banned at this point)

  10. Validate that the user is banned. Refreshing the project page should sign the user out and a message is shown in the login page

    📸 Screen_Shot_2022-10-20_at_2.18.23_PM

  11. Using the admin user (root), unban the user

    📸 Screen_Shot_2022-10-20_at_2.20.23_PM

  12. Using the developer/maintainer user, create a personal access token with api scope

  13. In a command line, download an archive of the first project (fill in <your_access_token> and <project_1_id> with the correct values)

    > curl -H 'Cache-Control: no-cache' --header "PRIVATE-TOKEN: <your_access_token>" "http://localhost:3000/api/v4/projects/<project_1_id>/repository/archive" --output -

  14. Validate that the download works as expected. You should see some gibberish in your command line like �kO�H���+������A�4p�A�"�PA�

  15. Download an archive of the second project (fill in <your_access_token> and <project_2_id> with the correct values)

    > curl -H 'Cache-Control: no-cache' --header "PRIVATE-TOKEN: <your_access_token>" "http://localhost:3000/api/v4/projects/<project_2_id>/repository/archive" --output -

  16. Validate that the download work does not work. You should see {"message":"403 Forbidden - You are not allowed to download code from this project."}

  17. Using the admin user (root), validate that the user is banned

    📸 Screen_Shot_2022-10-20_at_2.20.23_PM

Validate user is banned from a namespace when exceeding projects download threshold

  1. Turn off the feature flag for application-wide projects download throttling. Otherwise, that will take precedence 
    $ rails console
> Feature.disable(:git_abuse_rate_limit_feature_flag)
  2. Turn on the following feature flags: 
    $ rails console
> Feature.enable(:limit_unique_project_downloads_per_namespace_user)
  3. Using a normal (non-admin) user (owner), create a private top-level group and two projects under the group
  4. As owner, set the settings for the feature. Go to the group's page -> Settings -> Reporting
    📸 Screen_Shot_2022-10-20_at_2.42.57_PM
  5. Add a user (user) as a developer/maintainer to the group
  6. Using user from the previous step, download an archive of the first project
    📸 Screen_Shot_2022-10-20_at_1.58.25_PM
  7. Validate that the download works as expected
  8. Download an archive of the second project
  9. Validate that the download does not work (user is banned at this point)
  10. Validate that the user is banned from the namespace. Refreshing the project page return 404
  11. Using owner, validate that the user is banned from the namespace
    📸 Screen_Shot_2022-10-20_at_2.45.47_PM
  12. Using owner, unban the user
    📸 Screen_Shot_2022-10-20_at_2.45.47_PM
  13. Using user, create a personal access token with api scope
  14. In a command line, download an archive of the first project (fill in <your_access_token> and <project_1_id> with the correct values) 
    > curl -H 'Cache-Control: no-cache' --header "PRIVATE-TOKEN: <your_access_token>" "http://localhost:3000/api/v4/projects/<project_1_id>/repository/archive" --output -
  15. Validate that the download works as expected. You should see some gibberish in your command line like �kO�H���+������A�4p�A�"�PA�
  16. Download an archive of the second project (fill in <your_access_token> and <project_2_id> with the correct values) 
    > curl -H 'Cache-Control: no-cache' --header "PRIVATE-TOKEN: <your_access_token>" "http://localhost:3000/api/v4/projects/<project_2_id>/repository/archive" --output -
  17. Validate that the download work does not work. You should see {"message":"403 Forbidden - You are not allowed to download code from this project."}
  18. Using owner, validate that the user is banned
    📸 Screen_Shot_2022-10-20_at_2.45.47_PM

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Eugie Limpin

Merge request reports