Why did Splunk stop collecting syslog logs?

[lorder]

I installed Splunk last week, and I'm only collecting data (syslog) from one source.

Data stopped being collected this morning. I use Wireshark on the source server and Splunk, and I see that syslog are coming and going, but I don't see logs in Splunk. Latest event 3 hours ago.

License: Trial license group
License expiration Nov 17, 2018 4:04:30 PM

Licensed daily volume 500 MB

Volume used today 121 MB (24.135% of quota)

OS Windows 10 (Microsoft Windows [Version 10.0.16299.15])
SPLUNK Version:7.1.3 Build:51d9cac7b837

[mstjohn_splunk]

hi @lorder,

Could you give us some more context on this issue? For instance, as @dauren_akilbekov said, have you documented any errors that you could post? The more information you provide the community, the better chance you have of getting your question answered.

Thanks for posting!

[JDukeSplunk]

You should read or watch this excellent session from .conf 2017 - it was a very well attended session. This will give you a best practice syslog server to collect the logs:

http://conf.splunk.com/sessions/2017-sessions.html#search=critical%20syslog%20tricks&
https://conf.splunk.com/files/2017/slides/the-critical-syslog-tricks-that-no-one-seems-to-know-about...

[dauren_akilbeko]

Are you seeing errors at index=_internal source splunkd?

[lorder]

I use "index=_internal log_level=ERROR" and last eerors is:

09-20-2018 16:40:21.585 +0500 ERROR KVStoreBulletinBoardManager - KV Store changed status to failed. KVStore process terminated.

09-20-2018 16:40:21.584 +0500 ERROR KVStoreBulletinBoardManager - KV Store process terminated abnormally (exit code 14, status exited with code 14). See mongod.log and splunkd.log for details.

09-20-2018 16:40:21.568 +0500 ERROR MongodRunner - mongod exited abnormally (exit code 14, status: exited with code 14) - look at mongod.log to investigate.

2018-09-20 11:53:28,490 ERROR [5ba0dbbf9d126fbfbf240] root:130 - ENGINE: Handler for console events already off.