(feat): Replicate comments into OpenSearch and Vitess #2560

Ticket(s) / Related Merge Requests

#2560 (closed)

Summary of Changes

This MR introduces a new runner that subscribes to the entities-ops Pulsar topic and replicates comment data from Cassandra into Vitess and OpenSearch using the agreed upon schema.

Testing Considerations

First, create the table in MySQL:

CREATE TABLE IF NOT EXISTS minds_comments (
    guid bigint,
    entity_guid bigint,
    owner_guid bigint,
    parent_guid bigint REFERENCES minds_comment(guid),
    parent_depth int,
    body text,
    attachments json,
    mature boolean,
    edited boolean,
    spam boolean,
    deleted boolean,
    `enabled` boolean,
    group_conversation boolean,
    access_id bigint,
    time_created timestamp DEFAULT CURRENT_TIMESTAMP,
    time_updated timestamp DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    PRIMARY KEY (guid)
) ENGINE = InnoDB;

Next, the index in a local instance of Elasticsearch or OpenSearch

{
    "mappings": {
        "properties": {
            "@timestamp": {
                "type": "date"
            },
            "updated_at": {
                "type": "date"
            },
            "guid": {
                "type": "long"
            },
            "entity_guid": {
                "type": "long"
            },
            "owner_guid": {
                "type": "long"
            },
            "parent_guid": {
                "type": "long"
            },
            "parent_depth": {
                "type": "integer"
            },
            "body": {
                "type": "text"
            },
            "mature": {
                "type": "boolean"
            },
            "edited": {
                "type": "boolean"
            },
            "spam": {
                "type": "boolean"
            },
            "deleted": {
                "type": "boolean"
            },
            "is_enabled": {
                "type": "boolean"
            },
            "access_id": {
                "type": "long"
            },
            "group_conversation": {
                "type": "boolean"
            }
        }
    }
}

Start the runner with:

php cli.php EventStreams --subscription=Core\\Comments\\CommentOpsEventStreamsSubscription

Now, create a post and comments. You should see the tables and index be populated.

Note: This is currently in Sandbox as well.

Deployment Considerations

The above table and index should be create prior to deployment. The Vitess ACL will need to be updated.

Regression Scope

Comments manager was updated to allow for skipping the URN cache.

Platform Affected (web, mobile, etc)

Both

Developer Testing Completed

Tested that creating and updating comments updates the appropriate table and index.

Screenshots / Screen Recording

Screencast_from_05-08-2023_11_49_55_PM

Does this impact

• Localization
• Dark/light mode
• Guest mode

Definition of Done Checklist

• The Acceptance Criteria has been met
• Code is tested: Testing includes unit/spec, E2E/automated and manual testing
• Merge requests description has been filled out