Knowledge Management

Problem when updating the KV Store -- some filtered events are lost

Path Finder

Hi all,

We have about 15 Kvstores running ok but sometimes I detect that we had a update problem because we don't have all the filtered events there, we lose some... And we have to reload all the kvstore to fix it and again we have a fresh and updated kvstore version.
But I don't know why we have this problem, really, I can read this kind of error in _internal logs:

2017-09-19T12:17:33.254Z I COMMAND [conn148894] getmore local.oplog.rs query: { ts: { $gte: Timestamp 1505820444000|496 }, $or: [ { ns: { $regex: "^s_." }, $or: [ { op: { $in: [ "i", "u" ] }, o.user: "nobody" }, { op: "d" }, { op: "c", $or: [ { o.create: { $exists: 1 } }, { o.createDatabase: { $exists: 1 } }, { o.drop: { $exists: 1 } }, { o.dropDatabase: { $exists: 1 } } ] } ] }, { op: "c", ns: "admin.$cmd", o.renameCollection: { $exists: 1 }, o.to: { $regex: "^s." } }, { ts: Timestamp 1505820444000|496 } ] } cursorid:27155961562 ntoreturn:0 keyUpdates:0 writeConflicts:0 exception: getMore executor error: UnknownError no details available code:17406 numYields:0 nreturned:1 reslen:103 locks:{ Global: { acquireCount: { r: 4 } }, MMAPV1Journal: { acquireCount: { r: 4 } }, Database: { acquireCount: { r: 2 } }, oplog: { acquireCount: { R: 2 }, acquireWaitCount: { R: 2 }, timeAcquiringMicros: { R: 3536 } } } 1ms

Have you seen that before?
We upgrade Kv stores every 5 minutes and a couple of them are very big (+770K lines)...
Perhaps Splunk/mongodb is still upgrading a KV Store when we try to update it again?

Thanks a lot for your comments
Javier.

0 Karma
1 Solution

Path Finder

Yes, I was right, it was a problem with our update frecuency. I've increased all the updates to every 15 minutes or so and we don't have errors and we are not losing events anymore.

Thanks!

View solution in original post

0 Karma

Path Finder

Yes, I was right, it was a problem with our update frecuency. I've increased all the updates to every 15 minutes or so and we don't have errors and we are not losing events anymore.

Thanks!

View solution in original post

0 Karma