Skip to content

Add reporting/flagging of messages

Eric Eastwood requested to merge feature/report-users into develop

Add reporting/flagging of messages. Revival of https://github.com/troupe/gitter-webapp/pull/2564

  • Add a report option to messages (UI) that calls the API to record the report
  • API endpoint
    • Record report to Mongo collection chatmessagereports (perhaps redis in the future like we do for spam messages)
      • sent { type: Date, "default": Date.now }: When the report was sent
      • weight: Number: Factor of how much we consider of the individual report
      • reporterUserId: ObjectId: User who made the report
      • messageId: ObjectId: Offending bad message ID
      • messageUserId: ObjectId: Message author fromUserId (for an easy index when summing)
      • text: String: Message text at time of reporting
    • Count number of reports and check if over threshold. If over threshold, mark as bad user(hellban and if new user, delete messages)
  • Deduplicate reports from a user on each message (only allow one report to be counted for a given user against a message)
  • Weight calculations, baseWeight * userAgeWeight * messageAgeWeight
    • baseWeight (room admins have more weight)
      • Normal user: 1
      • Admin of room: 2.5
    • userAgeWeight (new users have little effect)
      • < 2 days: 0
      • < 2 weeks: 0.15
      • < 2 months: 0.3
      • < 6 months: 0.6
      • >= 6 months: 1
    • messageAgeWeight (old messages are hard to delete)
      • < 2 days: 1
      • < 21 days: 0.5
      • >= 21 days: 0
    • We want to avoid a troll insta-kill: A troll could create a room, add some people or mention them to come join, people respond "please stop, I'm not interested", and the troll could report.
      • We could also factor in room created date.
  • Sum calculations:
    • Report sum threshold to indicate bad user/message (escalation levels):
      • 2 to delete individual message
      • 5 to hellban and if new user, clear all messages
    • How do we handle a user marking each bad message? How is the report sum calculated?
      • Only count 1 report from a given user against another user
    • Only sum reports within the last 5 days
  • Add a dashboard that is only accessible to staff users, /-/admin/chat-message-reports

Testing locally

  • mongo localhost:27017
  • db.chatmessagereports.remove({})
  • .
    db.chatmessagereports.insertOne({ 
      "fromUserId": new ObjectId(),
      "messageId": new ObjectId(),
      "text" : "fake-report",
      "messageUserId": ObjectId("58f14d5ec72b1fa342f7e259"),
      "weight" : 2.5,
    });
  • set logging:level=info&&npm run dev

$ mklink "C:\Users\MLM\Documents\GitLab\gitter-secrets\webapp\env" "C:\Users\MLM\Documents\GitLab\gitter-secrets\env"
$ @FOR /f "tokens=*" %i IN ('bash /c/Users/MLM/Documents/GitLab/gitter-secrets/webapp/env test') DO @%i
$ npm run mocha -- test/api-tests/chat-api-tests.js

Todo

  • Only clear all messages for new users
  • Add basic dashboard to review reports
  • Add some docs around what messages should be reported

Future ideas

  • Add actions to report dashboard: hellban, delete message, see more messages from user, delete all messages from user
  • Audit/review reports
    • Factor in past good reports (from reviews) into users baseWeight
    • Allow the community to review reports. They can only review reports from rooms they can acccess
  • Add list of people who were banned
  • Graph of reports (we can get this in Datadog with the stats though)

Closes https://gitlab.com/gitlab-org/gitter/webapp/issues/954

Edited by Eric Eastwood

Merge request reports