Skip to content

Track spam scores for users

Ian Anderson requested to merge track-spam-scores into master

What does this MR do and why?

Describe in detail what your merge request does and why.

Track spam scores for users

The changes in this MR will save ML inference scores for users each time they attempt to create a "spammable" object. This score will be used to identify users abusing the product to publish spam and trigger automated actions against them.

This functionality is hidden behind the user_spam_scores feature flag which must be enabled to start tracking spam scores.

related issue: https://gitlab.com/gitlab-org/modelops/anti-abuse/team-tasks/-/issues/251

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

  1. Ensure you are logged into the GitLab docker registry
    docker login registry.gitlab.com
  2. Run the spamcheck service locally
    docker run --rm -p 8001:8001 registry.gitlab.com/gitlab-org/gl-security/security-engineering/security-automation/spam/spamcheck:1.7.2
  3. Enable spamcheck in your local GDK instance by navigating to admin/application_settings/reporting.
    • Enable spamcheck
    • Set the URL of the external Spam Check endpoint to grpc://localhost:8001 image
  4. As a regular user, create an issue in a public project (i.e. gitlab-org/gitlab-test).
  5. You should see logs from the spamcheck service that the issue was checked for spam.
  6. In the rails console look at the spam score for the user you used to create the issue. Since the feature has not been enabled the score should be 0.0
    [26] pry(main)> u = User.find(46)
    => #<User id:46 @imander>
    [27] pry(main)> u.spam_score
    => 0.0
  7. Enable the user_spam_scores feature flag.
    [1] pry(main)> Feature.enable(:user_spam_scores)
  8. Edit the description of the previously created issue to trigger another spam check.
  9. The user's spam score should now be updated.
    [28] pry(main)> u.spam_score
    => 0.1867965643276293e-9
    [29] pry(main)>

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Ian Anderson

Merge request reports