Fix Matrix import process not working with ghosted users

Fix Matrix import process not working with ghosted users

See #2609 (comment 1239203705)

Follow-up to !2313 (merged)

Part of #2609 (closed)

Problem

2023-01-13T21:15:06.864Z - error: Error occurred while backfilling events for opts.uri=MadLittleMods/import-test-with-ghosts gitterRoomId=63c1bfce85441f66183d5bbe: StatusError: sendEventAtTimestmap({ matrixRoomId: !jumHrTJrqHzpvLQVdd:my.matrix.host }) failed 403: {"errcode":"M_FORBIDDEN","error":"Application service has not registered this user (@ghost~5f762ffe986e461e663059f0-5f762ffe986e461e663059f0:my.matrix.host)"}
    at _sendEventWrapper (C:\Users\MLM\Documents\GitLab\webapp\modules\matrix-bridge\lib\matrix-utils.js:888:15)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async MatrixUtils.sendEventAtTimestmap (C:\Users\MLM\Documents\GitLab\webapp\modules\matrix-bridge\lib\matrix-utils.js:921:17)
    at async C:\Users\MLM\Documents\GitLab\webapp\modules\matrix-bridge\lib\gitter-to-matrix-historical-import.js:333:27

The problem is that we store the MXID as @ghost~5f762ffe986e461e663059f0-5f762ffe986e461e663059f0:my.matrix.host for ghosts but matrix-appservice-bridge registers the user as @ghost=7e5f762ffe986e461e663059f0-5f762ffe986e461e663059f0:my.matrix.host with escaped characters (notice the ~ replaced with =7e)

Reproduction steps

Reset:

set DEBUG=gitter:scripts:*&&set matrix__bridge__applicationServicePort=9001&&node ./scripts/utils/reset-matrix-historical-room-bridging.js --uri MadLittleMods/import-test-with-ghosts

set DEBUG=gitter:scripts:*&&set matrix__bridge__applicationServicePort=9001&&node ./scripts/utils/reset-matrix-room-bridging.js --uri MadLittleMods/import-test-with-ghosts

Setup:

1. Send some messages in the room
2. Delete and ghost that user
3. Remove the bridged Matrix user entry

   $ mongo gitter
   db.matricesbridgeduser.remove({ userId: ObjectId('5f762ffe986e461e663059f0') })

Run the failing import:

set DEBUG=gitter:scripts:*,gitter:app:matrix-bridge:gitter-to-matrix-historical-import,gitter:scripts-debug:events-imported&&set matrix__bridge__applicationServicePort=9001&&node ./scripts/utils/gitter-to-matrix-historical-import-one-room.js --uri MadLittleMods/import-test-with-ghosts

Dev notes

set DEBUG=gitter:scripts:*,gitter:app:matrix-bridge:gitter-to-matrix-historical-import,gitter:scripts-debug:events-imported&&set matrix__bridge__applicationServicePort=9001&&node ./scripts/utils/reset-mxids-for-ghosted-users.js --concurrency 1

Todo

• Write script to reset all Matrix bridged users for any ghosts
  • Run ./scripts/utils/reset-mxids-for-ghosted-users.js in production
• Update getOrCreateMatrixUserByGitterUserId to give compatible MXID to ghosted users (no ~)