Small fixes to the Matrix backfilling history script pt. 2
Small fixes to the Matrix backfilling history script pt. 2
Follow-up to !2313 (merged)
Part of #2609 (closed)
Dev notes
Error spawns from mongodb-core
-> lib/connection/connection.js
connection 10 to mongo-replica-01:27017 timed out, stack=MongoError: connection 10 to mongo-replica-01:27017 timed out
at Function.MongoError.create (/opt/gitter/gitter-webapp-staging-old/node_modules/mongoose/node_modules/mongodb-core/lib/error.js:29:11)
at Socket.<anonymous> (/opt/gitter/gitter-webapp-staging-old/node_modules/mongoose/node_modules/mongodb-core/lib/connection/connection.js:176:20)
at Object.onceWrapper (events.js:421:28)
at Socket.emit (events.js:315:20)
at Socket.EventEmitter.emit (domain.js:467:12)
at Socket._onTimeout (net.js:483:8)
at listOnTimeout (internal/timers.js:554:17)
at processTimers (internal/timers.js:497:7), name=MongoError
Are we getting timeouts because we're hitting max connections in Mongo?
idk
via https://dba.stackexchange.com/questions/73770/how-to-limit-the-connections-per-host-in-mongodb
> db.serverStatus().connections
{
"current" : 314,
"available" : 50886,
"totalCreated" : NumberLong(468146586)
}
via https://stackoverflow.com/a/51804590/796832
> db.adminCommand({getParameter:"*"})
{
// [...]
"connPoolMaxConnsPerHost" : 200,
"connPoolMaxShardedConnsPerHost" : 200,
// The default `cursorTimeoutMillis` is `600000ms` (10 minutes), https://www.mongodb.com/docs/manual/reference/parameters/#mongodb-parameter-param.cursorTimeoutMillis
"cursorTimeoutMillis" : 600000,
// [...]
}
Find connections by host/IP, via https://stackoverflow.com/a/55493746/796832
> db.currentOp(true).inprog.reduce(
(accumulator, connection) => {
ipaddress = connection.client ? connection.client.split(":")[0] : "Internal";
accumulator[ipaddress] = (accumulator[ipaddress] || 0) + 1;
accumulator["TOTAL_CONNECTION_COUNT"]++;
return accumulator;
},
{ TOTAL_CONNECTION_COUNT: 0 }
)
// Example return:
{
"TOTAL_CONNECTION_COUNT" : 331,
"192.168.253.72" : 8,
"192.168.254.42" : 17,
"127.0.0.1" : 3,
"192.168.248.66" : 2,
"11.178.12.244" : 2,
"Internal" : 41,
"3.100.12.33" : 86,
"11.148.23.34" : 168,
"81.127.34.11" : 1,
"84.147.25.17" : 3
}
https://www.mongodb.com/docs/v2.4/reference/configuration-options/#maxConns
maxIncomingConnections
maxConns
No cursor timeout
We can't use .addCursorFlag('noCursorTimeout', true)
because this was introduced in mongoose@4.11.0
and we're using mongoose@4.6.8
and blocked on upgrading.
- Docs for
mongoose@4.x
: https://mongoosejs.com/docs/4.x/docs/api.html#cursor_QueryCursor_QueryCursor-addCursorFlag - https://www.mongodb.com/docs/manual/reference/method/cursor.noCursorTimeout/
Versions:
-
mongoose@4.6.8
-
mongodb@2.2.11
: https://github.com/mongodb/node-mongodb-native/tree/V2.2.11 -
mongodb-core@2.0.13
: https://github.com/mongodb-js/mongodb-core/tree/V2.0.13
-
Edited by Eric Eastwood