Deployment Architecture

When KV Store got restarted

Master_Blaster
Engager

Hi,

We have a search head cluster of 8 members in which KV store is failing frequently. We used to start services manually.

I'd like to create a report which should contains when exactly  kv store got failed & when it got up. I am not sure in which logs we can find this info. 
Could anyone help me with the query for same ?
Thanks

 

Tags (1)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

I believe you want to search for "MongoDB starting"

index=_internal  sourcetype=mongod "MongoDB starting"
---
If this reply helps you, an upvote would be appreciated.
0 Karma

gjanders
SplunkTrust
SplunkTrust

In alerts for splunk admins https://splunkbase.splunk.com/app/3796/ I have an alert to detect a lack of logging from mongod.log so combined with richgalloway's answer this might work for you...

0 Karma

Master_Blaster
Engager

Unfortunately, the query doesn't help.  I see multiple entries of below message where we didn't do any actual restart.


2021-03-25T08:14:43.060Z I CONTROL [initandlisten] MongoDB starting : pid=18614 port=8191 dbpath=/opt/splunk/var/lib/splunk/kvstore/mongo 64-bit host=xxxxxx

0 Karma