## Goals - [x] Internal testing with Admins (March 26th) - [x] Test suite (MR passing) - [x] Legal clarification for illegal reports (@jotto141) - [x] Marketing text (@jotto141) - [x] Decision on legacy reports and transition - [x] Allow channel reports. Create limited state for ban appeals. Notify users when channel marked as NSFW. - [x] Add warning text for users opting-in to Jury: "I am 18 years or older and volunteer to participate in this jury. I acknowledge that I may be exposed to content not safe for work (NSFW) and understand that the purpose of the trial is to enforce the content policy outlined here." - [ ] Add warning text for removing a NSFW blur from a post: "I am 18+ and would like to see content that is not safe for work (NSFW). <view>" - [ ] Only users opted-in to NSFW should see NSFW tagged boosts. - [ ] Update Boost reasons to Accept, Mark NSFW, Reject/Report - [ ] Show ledger of past results of your juries in settings - [ ] Apply score of +25 reward for participating in jury - [x] Build leasing system and time limit for users to join jury. If they do not join in time, jury request randomly sent to next active user [Moderation_Splash_Page.sketch](/uploads/920b3cbd1501a58d8628906ff44439be/Moderation_Splash_Page.sketch) ## Report Options Illegal (1) (not to appeal jury) - Terrorism - Paedophilia - Extortion - Fraud - Revenge Porn - Sex trafficking NSFW (2) - Nudity (1) - Pornography (2) - Profanity (3) - Violence and Gore (4) - Race, Religion, Gender (5) - Other (6) - needed for all legacy explicit tags Encourages or incites violence (3) (not to appeal jury) Harassment (4) Personal and confidential information (5) (not to appeal jury) Impersonates (7) (not to appeal jury) Trademark infringement (15) (not to appeal jury) Spam (8) (sometimes to appeal jury) Infringes my copyright (DMCA notice) (10) (not to appeal jury) Incorrect use of hashtags (Content only) (12) Token manipulation (16) (not to appeal jury) Malware (Content only) (13) (not to appeal jury) Strikes (Channel/System only) (14) ## Making a Report **Schema:** entity_urn reason_code subreason_code timestamp reporter_guid urn:activity:123 2 5 now 456 The below schemas where replaced by Reports (Cases) below ```sql CREATE TABLE moderation_reports ( entity_urn text, timestamp timestamp, reason_code float, sub_reason_code float, reporter_guid bigint, entity_owner_guid bigint, PRIMARY KEY(entity_urn, reason_code, sub_reason_code, timestamp, reporter_guid) ); INSERT INTO moderation_reports( entity_urn, timestamp, reason_code, sub_reason_code, reporter_guid ) VALUES ( 'urn:activity:123', 1556629964718, 2, 5, 456 ); SELECT * FROM moderation_reports WHERE entity_urn='urn:activity:123' AND reason_code=2 AND sub_reason_code=5 AND timestamp>'2019-04-30' ``` A request should be made to see if a ‘case’ exists. If not then one should be created. If a case does exists, or is already closed and of a valid ‘multiple report’ type (eg. users), then a new case should be opened. ## Report Cases Certain types of reports require report cases (eg. users) to resolve scenarios of when an overturned appeal could give lifetime immunity. For example, If a channel won an appeal against “Threatens, harrsasses or bullies”, but then in a few months is found in violation of the terms, an additional appeal would then have to be created. **Schema:** entity_urn reason_code subreason_code timestamp state urn:activity:123 2 5 TS ‘appealed’ ```sql CREATE TABLE moderation_reports ( entity_urn text, reason_code tinyint, sub_reason_code decimal, timestamp timestamp, state text, reports SET<bigint>, initial_jury MAP<bigint, boolean>, appeal_jury MAP<bigint, boolean>, user_hashes SET<text>, entity_owner_guid bigint, appeal_note text, state_changes MAP<text, timestamp>, uphold boolean, PRIMARY KEY(entity_urn, reason_code, sub_reason_code, timestamp) ) WITH CLUSTERING ORDER BY (reason_code ASC, sub_reason_code ASC, timestamp DESC) CREATE MATERIALIZED VIEW moderation_reports_by_state AS SELECT * FROM moderation_reports WHERE state IS NOT NULL AND reason_code IS NOT NULL AND sub_reason_code IS NOT NULL AND timestamp IS NOT NULL PRIMARY KEY (state, timestamp, entity_urn, reason_code, sub_reason_code) WITH CLUSTERING ORDER BY (timestamp DESC, entity_urn ASC, reason_code ASC, sub_reason_code ASC); CREATE MATERIALIZED VIEW moderation_reports_by_entity_owner_guid AS SELECT * FROM moderation_reports WHERE entity_owner_guid IS NOT NULL AND reason_code IS NOT NULL AND sub_reason_code IS NOT NULL AND timestamp IS NOT NULL PRIMARY KEY (entity_owner_guid, timestamp, entity_urn, reason_code, sub_reason_code) WITH CLUSTERING ORDER BY (timestamp DESC, entity_urn ASC, reason_code ASC, sub_reason_code ASC); SELECT state FROM moderation_reports WHERE entity_urn='urn:activity:123' AND reason_code=5 AND sub_reason_code=5 ORDER BY reason_code DESC, sub_reason_code DESC, timestamp DESC LIMIT 1; UPDATE moderation_reports SET state='reported', reports += {456}, user_hashes += {'mark'} WHERE entity_urn= 'urn:activity:123' AND reason_code=2 AND sub_reason_code=5 AND timestamp=1556629964719; UPDATE moderation_reports SET reports += {456}, user_hashes += {} WHERE entity_urn= 'urn:activity:1234' AND reason_code=2 AND sub_reason_code=5 AND timestamp=1556629964719; select * from moderation_reports; ``` Valid case states are: reported awaiting_initial_decision initial_jury_decided appealed awaiting_appeal_decision appeal_jury_decided In order to find the currently opened cases for an entity, you’d simply sort my timestamp. Cases will be indexed in Elasticsearch too. ## Jury System ~~Jury act on cases, not on the entities themselves. This allows for multiple decisions to be made on a post.~~ ## Strike System **Strike Policy:** Users will receive a strike for the following term violations. Users will be notified about the strike, which term was violated, and which piece of content was in violation. All strikes can be appealed to a jury of Minds community members. Individual strikes expire after 3 months. 10 cumulative strikes result in a ban, regardless of the expiration rule. **NSFW (three strikes for each individual NSFW option)** Strike 1 = Warning Strike 2 = 2nd Warning Strike 3 = Channel marked NSFW for as long as they have 3 strikes **Harassment** Strike 1 = Warning Strike 2 = 2nd Warning Strike 3 = Ban **Spam*** Strike 1 = Warning Strike 2 = 2nd Warning Strike 3 = Ban Spam may result in an immediate ban if determined to be malicious or by use of bot by Minds admins **Schema:** user_guid timestamp reason_code subreason_code case_urn 456 now 2 5 urn:case:(urn:activity:123)-2.5-1556629964719 ```sql CREATE TABLE moderation_strikes ( user_guid bigint, timestamp timestamp, reason_code tinyint, sub_reason_code decimal, report_urn text, PRIMARY KEY(user_guid, reason_code, sub_reason_code, timestamp) ); INSERT INTO moderation_strikes( user_guid, timestamp, reason_code, sub_reason_code, case_urn ) VALUES ( 456, 1557629964729, 2, 5, 'urn:moderation:case:(urn:activity:123)-2.5-1556629964719' ); ``` On the 3rd strike, an automatic decision will be made by the system. This is so the user can appeal. ## Channel Level Reporting An important distinction needs to be made between ‘strike’ enacted decisions and channel level reports. For example, should a channel be able to appeal the decision of their 3rd strike that imposed the ban on their channel? Would the strike enacted decision be a reason of ‘3 strikes’ or would it be the reason of the strike? ## Juror Integrity * If a juror votes against the consensus of the jury 10 times, they are no longer able to be a juror. [not implemented] * A juror cannot vote on their own reports or content. ## Elastic Search Querying Elasticsearch will be the primary querying system and cassandra will be used for the core storage and reading of reports. Delegates will assist in the copying/cloning of records to elasticsearch. ## Immediate Bans **Immediate Ban Policy:** Users will be immediately banned for each of the following violations. Users will be notified about the ban and which term was violated, but they will not be able to see the content that was in violation as it will have to be removed from Minds. Appeals on immediate bans will be reviewed by the Minds admins and not a jury due to the nature of the content. - Illegal (terrorism, fraud, sex trafficking, child pornography, revenge porn) - Personal and confidential information (doxxing) - Malware - Impersonates - Token Manipulation - Encourages or incites violence ## Summons TBD: TBD ```sql CREATE TABLE minds.moderation_summons ( report_urn text, jury_type text, juror_guid bigint, status text, expires int, PRIMARY KEY (report_urn, jury_type, juror_guid) ); ```