Try again and resume if the import fails (like a cursor timeout) - Small fixes to the Matrix backfilling history script pt. 6
Small fixes to the Matrix backfilling history script pt. 6: Try again and resume if the import fails (like a cursor timeout)
Follow-up to !2313 (merged)
Part of #2609 (closed)
Error occurred while backfilling events: MongoError: Cursor not found, cursor id: 240894090023
at Function.MongoError.create (/opt/gitter/gitter-webapp-staging/node_modules/mongoose/node_modules/mongodb-core/lib/error.js:31:11)
at /opt/gitter/gitter-webapp-staging/node_modules/mongoose/node_modules/mongodb-core/lib/connection/pool.js:483:72
at authenticateStragglers (/opt/gitter/gitter-webapp-staging/node_modules/mongoose/node_modules/mongodb-core/lib/connection/pool.js:429:16)
at Connection.messageHandler (/opt/gitter/gitter-webapp-staging/node_modules/mongoose/node_modules/mongodb-core/lib/connection/pool.js:463:5)
at Socket.<anonymous> (/opt/gitter/gitter-webapp-staging/node_modules/mongoose/node_modules/mongodb-core/lib/connection/connection.js:309:22)
at Socket.emit (events.js:315:20)
at Socket.EventEmitter.emit (domain.js:467:12)
at addChunk (internal/streams/readable.js:309:12)
at readableAddChunk (internal/streams/readable.js:284:9)
at Socket.Readable.push (internal/streams/readable.js:223:10)
at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
Todo
-
Add some sort of abort mechanism for the currently running lanes before we restart any import process. We need to be careful not to have two processes importing the same room. -
Add a stat to the dashboard to indicate how many times the import process has been restarted
Edited by Eric Eastwood