Skip to content

Preemptively resume cursor if it will timeout soon - Small fixes to the Matrix backfilling history script pt. 8

Eric Eastwood requested to merge backfill-messages-to-matrix-follow-up8 into develop

Small fixes to the Matrix backfilling history script pt. 8:

  • Preemptively resume cursor if it will timeout soon
  • Use a smaller batch size so we exercise the database connection more often to avoid stale cursors in the first place

Alternative to !2324 (closed)

Follow-up to !2313 (merged)

Part of #2609 (closed)

Error occurred while backfilling events: MongoError: Cursor not found, cursor id: 240894090023
    at Function.MongoError.create (/opt/gitter/gitter-webapp-staging/node_modules/mongoose/node_modules/mongodb-core/lib/error.js:31:11)
    at /opt/gitter/gitter-webapp-staging/node_modules/mongoose/node_modules/mongodb-core/lib/connection/pool.js:483:72
    at authenticateStragglers (/opt/gitter/gitter-webapp-staging/node_modules/mongoose/node_modules/mongodb-core/lib/connection/pool.js:429:16)
    at Connection.messageHandler (/opt/gitter/gitter-webapp-staging/node_modules/mongoose/node_modules/mongodb-core/lib/connection/pool.js:463:5)
    at Socket.<anonymous> (/opt/gitter/gitter-webapp-staging/node_modules/mongoose/node_modules/mongodb-core/lib/connection/connection.js:309:22)
    at Socket.emit (events.js:315:20)
    at Socket.EventEmitter.emit (domain.js:467:12)
    at addChunk (internal/streams/readable.js:309:12)
    at readableAddChunk (internal/streams/readable.js:284:9)
    at Socket.Readable.push (internal/streams/readable.js:223:10)
    at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
Edited by Eric Eastwood

Merge request reports