Add reporting/flagging of messages
Add reporting/flagging of messages. Revival of https://github.com/troupe/gitter-webapp/pull/2564
- Add a report option to messages (UI) that calls the API to record the report
- API endpoint
- Record report to Mongo collection
chatmessagereports
(perhaps redis in the future like we do for spam messages)-
sent { type: Date, "default": Date.now }
: When the report was sent -
weight: Number
: Factor of how much we consider of the individual report -
reporterUserId: ObjectId
: User who made the report -
messageId: ObjectId
: Offending bad message ID -
messageUserId: ObjectId
: Message authorfromUserId
(for an easy index when summing) -
text: String
: Message text at time of reporting
-
- Count number of reports and check if over threshold. If over threshold, mark as bad user(hellban and if new user, delete messages)
- Record report to Mongo collection
- Deduplicate reports from a user on each message (only allow one report to be counted for a given user against a message)
- Weight calculations,
baseWeight * userAgeWeight * messageAgeWeight
-
baseWeight
(room admins have more weight)- Normal user:
1
- Admin of room:
2.5
- Normal user:
-
userAgeWeight
(new users have little effect)-
< 2 days
:0
-
< 2 weeks
:0.15
-
< 2 months
:0.3
-
< 6 months
:0.6
-
>= 6 months
:1
-
-
messageAgeWeight
(old messages are hard to delete)-
< 2 days
:1
-
< 21 days
:0.5
-
>= 21 days
:0
-
- We want to avoid a troll insta-kill: A troll could create a room, add some people or mention them to come join, people respond "please stop, I'm not interested", and the troll could report.
- We could also factor in room created date.
-
- Sum calculations:
- Report sum threshold to indicate bad user/message (escalation levels):
-
2
to delete individual message -
5
to hellban and if new user, clear all messages
-
- How do we handle a user marking each bad message? How is the report sum calculated?
- Only count 1 report from a given user against another user
- Only sum reports within the last 5 days
- Report sum threshold to indicate bad user/message (escalation levels):
- Add a dashboard that is only accessible to staff users,
/-/admin/chat-message-reports
Testing locally
mongo localhost:27017
db.chatmessagereports.remove({})
- .
db.chatmessagereports.insertOne({ "fromUserId": new ObjectId(), "messageId": new ObjectId(), "text" : "fake-report", "messageUserId": ObjectId("58f14d5ec72b1fa342f7e259"), "weight" : 2.5, });
set logging:level=info&&npm run dev
$ mklink "C:\Users\MLM\Documents\GitLab\gitter-secrets\webapp\env" "C:\Users\MLM\Documents\GitLab\gitter-secrets\env"
$ @FOR /f "tokens=*" %i IN ('bash /c/Users/MLM/Documents/GitLab/gitter-secrets/webapp/env test') DO @%i
$ npm run mocha -- test/api-tests/chat-api-tests.js
Todo
-
Only clear all messages for new users -
Add basic dashboard to review reports -
Add some docs around what messages should be reported
Future ideas
- Add actions to report dashboard: hellban, delete message, see more messages from user, delete all messages from user
- Audit/review reports
- Factor in past good reports (from reviews) into users
baseWeight
- Allow the community to review reports. They can only review reports from rooms they can acccess
- Factor in past good reports (from reviews) into users
- Add list of people who were banned
- Graph of reports (we can get this in Datadog with the stats though)
Closes https://gitlab.com/gitlab-org/gitter/webapp/issues/954
Edited by Eric Eastwood