- 21 Jan, 2023 4 commits
-
-
Eric Eastwood authored
Just some extra Matrix room creation logging See merge request !2356
-
Eric Eastwood authored
-
Eric Eastwood authored
Resolve conflict where Matrix room exists with alias but we don't have it stored yet See merge request !2357
-
Eric Eastwood authored
(consistency problem) This solves another class of problems we are seeing with the Gitter -> Matrix import process: ``` Error occurred while backfilling events for opts.uri=undefined gitterRoomId=xxx: matrix-appservice-bridge/matrix-bot-sdk threw an error that looked more like a request object, see {"statusCode":400,"body":{"errcode":"M_ROOM_IN_USE","error":"Room alias already taken"},"headers":{"date":"Fri, 20 Jan 2023 07:55:14 GMT","content-type":"application/json","transfer-encoding":"chunked","connection":"close","cache-control":"no-cache, no-store, must-revalidate","access-control-allow-origin":"*","access-control-allow-methods":"GET, HEAD, POST, PUT, DELETE, OPTIONS","access-control-allow-headers":"X-Requested-With, Content-Type, Authorization, Date","strict-transport-security":"max-age=15724800; includeSubDomains"},"request":{"uri":{"protocol":"https:","slashes":true,"auth":null,"host":"gitter.ems.host","port":443,"hostname":"gitter.ems.host","hash":null,"search":null,"query":null,"pathname":"/_matrix/client/r0/createRoom","path":"/_matrix/client/r0/createRoom","href":"https://gitter.ems.host/_matrix/client/r0/createRoom"},"method":"POST","headers":{"Authorization":"<redacted>","Content-Type":"application/json","content-length":117}}} ``` Follow-up to !2313 Part of #2609
-
- 20 Jan, 2023 1 commit
-
-
Eric Eastwood authored
-
- 19 Jan, 2023 12 commits
-
-
Eric Eastwood authored
21.56.3
-
Eric Eastwood authored
-
Eric Eastwood authored
Previous MR where we added `/.well-known/matrix/server`: !2031 Part of https://github.com/matrix-org/matrix-ansible-private/issues/6361 and the Gitter sunset: https://github.com/vector-im/roadmap/issues/26
-
Eric Eastwood authored
-
Eric Eastwood authored
Add `/.well-known/matrix/client` for Matrix clients See merge request !2336
-
Eric Eastwood authored
Previous MR where we added `/.well-known/matrix/server`: !2031 Part of https://github.com/matrix-org/matrix-ansible-private/issues/6361 and the Gitter sunset: https://github.com/vector-im/roadmap/issues/26
-
Eric Eastwood authored
-
Eric Eastwood authored
Persist room ID's that failed to import along the way so we can review See merge request !2354
-
Eric Eastwood authored
Before we would only persist all the failures at the end but that means we can't debug anything while it's still running. And even if we can see the shell terminal output for the running import process, sometimes it disconnects and we lose the insight. Follow-up to !2313 Part of #2609
-
Eric Eastwood authored
-
Eric Eastwood authored
Matrix bridge doesn't need to have exclusive control over the namespaces in a self-service world See merge request !2335
-
Eric Eastwood authored
As part of the [sunset plan for Gitter](https://github.com/vector-im/roadmap/issues/26), we need to make the appservice not be exclusive over the namespaces defined in the appservice registration file so people don't run into errors when they self-service and try to create things. Part of https://github.com/matrix-org/matrix-hosted/issues/6979
-
- 18 Jan, 2023 16 commits
-
-
Eric Eastwood authored
-
Eric Eastwood authored
-
Eric Eastwood authored
Fix Matrix import process not working with usernames that contain tildes (~) See merge request !2353
-
Eric Eastwood authored
Example: `removed~foobar` Follow-up to !2333 !2313 Part of #2609
-
Eric Eastwood authored
-
Eric Eastwood authored
Better log out the requests that matrix-appservice-bridge/matrix-bot-sdk throws out as errors See merge request !2352
-
Eric Eastwood authored
We're only worrying about improving because that's where we drill down and investigate from more easily than the giant output from an import worker. Follow-up to !2313 Part of #2609 ### Before in `scripts/utils/gitter-to-matrix-historical-import-one-room.js` Before in `scripts/utils/gitter-to-matrix-historical-import-one-room.js` (yes, it doesn't print the error because it doesn't have an `err.stack` like errors should): ``` 2023-01-18T09:30:06.468Z - error: Error occurred while backfilling events for opts.uri=undefined gitterRoomId=xxx: ``` ### Before in `scripts/utils/gitter-to-matrix-historical-import-worker.js` Before in `scripts/utils/gitter-to-matrix-historical-import-worker.js` it printed a garbled mess with no indication which request or URL failed. ``` 2023-01-17T20:26:19.683Z - error: concurrentQueue: Failed to process itemId=xxx_eventsCount=4, httpVersionMajor=1, httpVersionMinor=1, httpVersion=1.1, complete=true, rawHeaders=Date,Tue, 17 Jan 2023 20:26:19 GMT,Content-Type,application/json,Transfer-Encoding,chunked,Connection,close,Cache-Control,no-cache, no-store, must-revalidate,Access-Control-Allow-Origin,*,Access-Control-Allow-Methods,GET, HEAD, POST, PUT, DELETE, OPTIONS,Access-Control-Allow-Headers,X-Requested-With, Content-Type, Authorization, Date,Strict-Transport-Security,max-age=15724800; includeSubDomains, rawTrailers=, aborted=false, upgrade=false, url=, method=null, statusCode=400, statusMessage=Bad Request, _consuming=true, _dumped=false, toJSON=function responseToJSON () { var self = this return { statusCode: self.statusCode, body: self.body, headers: self.headers, request: requestToJSON.call(self.request) } } ``` ### After in `scripts/utils/gitter-to-matrix-historical-import-one-room.js` Now we actually get the request URL and details of the response: ``` 2023-01-18T09:44:24.581Z - error: Error occurred while backfilling events for opts.uri=undefined gitterRoomId=xxx: matrix-appservice-bridge/matrix-bot-sdk threw an error that looked more like a request object, see {"statusCode":400,"body":{"errcode":"M_ROOM_IN_USE","error":"Room alias already taken"},"headers":{"date":"Wed, 18 Jan 2023 09:44:24 GMT","content-type":"application/json","transfer-encoding":"chunked","connection":"close","cache-control":"no-cache, no-store, must-revalidate","access-control-allow-origin":"*","access-control-allow-methods":"GET, HEAD, POST, PUT, DELETE, OPTIONS","access-control-allow-headers":"X-Requested-With, Content-Type, Authorization, Date","strict-transport-security":"max-age=15724800; includeSubDomains"},"request":{"uri":{"protocol":"https:","slashes":true,"auth":null,"host":"gitter.ems.host","port":443,"hostname":"gitter.ems.host","hash":null,"search":null,"query":null,"pathname":"/_matrix/client/r0/createRoom","path":"/_matrix/client/r0/createRoom","href":"https://gitter.ems.host/_matrix/client/r0/createRoom"},"method":"POST","headers":{"Authorization":"<redacted>","Content-Type":"application/json","content-length":127}}} 2023-01-18T09:44:24.583Z - warn: Waiting for this ongoing message send request to finish and bridged information stored... redis handler failed during shutdown: TypeError: Cannot read property 'activeMasterClient' of undefined ```
-
Eric Eastwood authored
21.56.2
-
Eric Eastwood authored
-
Eric Eastwood authored
As suggested by @dregad, https://gitter.im/gitter/gitter?at=63c7afc5fce37f655ff65000
-
Eric Eastwood authored
-
Eric Eastwood authored
Update homepage latest news block to point to Matrix migration blog post See merge request !2350
-
Eric Eastwood authored
As suggested by @dregad, https://gitter.im/gitter/gitter?at=63c7afc5fce37f655ff65000
-
Eric Eastwood authored
-
Eric Eastwood authored
Add a way to manually dump from a specific list of users See merge request !2349
-
Eric Eastwood authored
These are users we need to manually fix up since we found their entries to be messed up, See #2856 (comment 1243268761) Follow-up to !2318
-
- 17 Jan, 2023 7 commits
-
-
Eric Eastwood authored
-
Eric Eastwood authored
Fix typo in logging call preventing our skips from working See merge request !2348
-
Eric Eastwood authored
Follow-up to !2313 Part of #2609 The skips were originally added in: - !2345 - !2344
-
Eric Eastwood authored
-
Eric Eastwood authored
Fix old ORG_CHANNEL rooms with no `groupId` not importing See merge request !2346
-
Eric Eastwood authored
These should have been picked up and fixed way back when we migrated to `groupId` so we would never run into a room like this but I guess it was never caught. Let's just deal with it. Follow-up to !2313 Part of #2609 The import script was running into the following error: ```sh $ matrix__bridge__applicationServicePort=9001 NODE_ENV=prod ./scripts/utils/gitter-to-matrix-historical-import-one-room.js --uri foo/bar 17 Jan 08:33:48 - opening socket: ws://cube.prod.gitter:1080/1.0/event/put 2023-01-17T08:33:49.898Z - info: Setting up Matrix bridge 2023-01-17T08:33:49.915Z - info: Matrix bridge listening on port 9001 2023-01-17T08:33:49.915Z - info: Ensuring profile info is up-to-date for the Matrix bridge user mxid=@gitter-badger:gitter.im 2023-01-17T08:33:50.011Z - info: Starting import script for foo/bar (57580f83c2f0db084a1cca60) --> matrixHistoricalRoomId=!xxx:gitter.im (live matrixRoomId=!xxx:gitter.im) MatrixHttpClient (REQ-1) { errcode: 'M_USER_IN_USE', error: 'User ID already taken.' } 2023-01-17T08:33:50.245Z - error: Error occurred while backfilling events for opts.uri=foo/bar gitterRoomId=57580f83c2f0db084a1cca60: AssertionError [ERR_ASSERTION]: groupId=undefined unexpectedly does not exist for gitterRoomId=57580f83c2f0db084a1cca60 at MatrixUtils.ensureCorrectRoomState (./modules/matrix-bridge/lib/matrix-utils.js:367:5) at async MatrixUtils.ensureCorrectHistoricalMatrixRoomStateAfterImport (./modules/matrix-bridge/lib/matrix-utils.js:536:7) at async gitterToMatrixHistoricalImport (./modules/matrix-bridge/lib/gitter-to-matrix-historical-import.js:616:3) at async exec (./scripts/utils/gitter-to-matrix-historical-import-one-room.js:50:3) 2023-01-17T08:33:50.246Z - warn: Waiting for this ongoing message send request to finish and bridged information stored... redis handler failed during shutdown: TypeError: Cannot read property 'activeMasterClient' of undefined ``` ### Testing strategy ``` $ mongo gitter > db.troupes.insert({ "githubType" : "ORG", "userCount" : 9, "security" : null, "githubId" : 5990364, "uri" : "gitterhq-legacy-test-org-channel", "lcOwner" : "gitterhq-legacy-test-org-channel", "lcUri" : "gitterhq-legacy-test-org-channel", "__v" : 0, "sd" : { "type" : "GH_ORG", "members" : "GH_ORG_MEMBER", "admins" : "GH_ORG_MEMBER", "public" : false, "linkPath" : "gitterhq", "externalId" : 5990364 } }); > db.troupes.insert({ "githubType" : "ORG_CHANNEL", "lcUri" : "gitterhq-legacy-test-org-channel/no-group-id", "userCount" : 0, "ownerUserId" : null, "parentId" : db.troupes.findOne({ lcUri: "gitterhq-legacy-test-org-channel" })._id, "security" : "PUBLIC", "uri" : "gitterhq-legacy-test-org-channel/no-group-id", "lcOwner" : "gitterhq-legacy-test-org-channel", "bans" : [ ], "oneToOneUsers" : [ ], "providers" : [ ], "renamedLcUris" : [ ], "status" : "ACTIVE", "tags" : [ ], "sd" : { "type" : "GH_ORG", "members" : "INVITE", "admins" : "GH_ORG_MEMBER", "public" : false, "linkPath" : "gitterhq", "externalId" : 5990364 } }); ``` ``` matrix__bridge__applicationServicePort=9001 node ./scripts/utils/gitter-to-matrix-historical-import-one-room.js --uri gitterhq-legacy-test-org-channel/no-group-id ```
-
Eric Eastwood authored
Add room ID option to import one room script for easier debugging See merge request !2347
-