topic Re: Why did Splunk stop collecting syslog logs? in Getting Data In

Why did Splunk stop collecting syslog logs?

lorder — Thu, 20 Sep 2018 09:21:13 GMT

I installed Splunk last week, and I'm only collecting data (syslog) from one source.

Data stopped being collected this morning. I use Wireshark on the source server and Splunk, and I see that syslog are coming and going, but I don't see logs in Splunk. Latest event 3 hours ago.

License: Trial license group
License expiration Nov 17, 2018 4:04:30 PM

Licensed daily volume 500 MB

Volume used today 121 MB (24.135% of quota)

OS Windows 10 (Microsoft Windows [Version 10.0.16299.15])
SPLUNK Version:7.1.3 Build:51d9cac7b837

Re: Why did Splunk stop collecting syslog logs?

dauren_akilbeko — Thu, 20 Sep 2018 10:45:38 GMT

Are you seeing errors at index=_internal source splunkd?

Re: Why did Splunk stop collecting syslog logs?

JDukeSplunk — Thu, 20 Sep 2018 17:52:48 GMT

You should read or watch this excellent session from .conf 2017 - it was a very well attended session. This will give you a best practice syslog server to collect the logs:

http://conf.splunk.com/sessions/2017-sessions.html#search=critical%20syslog%20tricks&
https://conf.splunk.com/files/2017/slides/the-critical-syslog-tricks-that-no-one-seems-to-know-about.pdf

Re: Why did Splunk stop collecting syslog logs?

mstjohn_splunk — Thu, 20 Sep 2018 20:51:32 GMT

hi @lorder,

Could you give us some more context on this issue? For instance, as @dauren_akilbekov said, have you documented any errors that you could post? The more information you provide the community, the better chance you have of getting your question answered.

Thanks for posting!

Re: Why did Splunk stop collecting syslog logs?

lorder — Tue, 29 Sep 2020 21:18:08 GMT

I use "index=_internal log_level=ERROR" and last eerors is:

09-20-2018 16:40:21.585 +0500 ERROR KVStoreBulletinBoardManager - KV Store changed status to failed. KVStore process terminated.

09-20-2018 16:40:21.584 +0500 ERROR KVStoreBulletinBoardManager - KV Store process terminated abnormally (exit code 14, status exited with code 14). See mongod.log and splunkd.log for details.

09-20-2018 16:40:21.568 +0500 ERROR MongodRunner - mongod exited abnormally (exit code 14, status: exited with code 14) - look at mongod.log to investigate.

2018-09-20 11:53:28,490 ERROR [5ba0dbbf9d126fbfbf240] root:130 - ENGINE: Handler for console events already off.