topic Re: How do you troubleshoot missing windows event logs? in Getting Data In

How do you troubleshoot missing windows event logs?

ABasit10 — Thu, 07 Mar 2019 21:11:31 GMT

I have been noticing that some windows event logs are not appearing in the Splunk search. For example the event code for windows restart is 1074 but whenever I use the search string below, I do not get any results back within the specific time period. I know for a fact that the event codes are not blacklisted in the configuration files as I can view older logs for the same event code. When checking the logs in Event Viewer, I can also see the logs for the windows restart event. I am not sure why the event is logged sometimes but other times it isn't. This issue isn't exclusive to a particular windows event as I have noticed missing logs for other events as well.

Query: index=* host=DC* EventCode=1074

Help with troubleshooting the issue would be greatly appreciated.

Re: How do you troubleshoot missing windows event logs?

nickhills — Fri, 08 Mar 2019 11:52:39 GMT

Do you have current_only=1 set on your windows forwarders?

This setting results in ONLY events generated while Splunk is running being sent.
This means that if you shutdown your host, any logs generated between the time the Splunk process stops and restarts will never be indexed.

Re: How do you troubleshoot missing windows event logs?

ABasit10 — Fri, 08 Mar 2019 20:41:23 GMT

Thanks for the response. I have validated that current_only=0 is the value set on the forwarders. The strangest thing is that the event is sometimes logged and other time is isn't. For testing, i had manually restarted the host to see if the event code 1074 was being logged by Splunk. I wasn't able to find any event between for that time frame however, i saw a windows host restart event on a different day with a different time stamp.

I am not sure why some events for this particular event code (1074) are being logged and others aren't.

Re: How do you troubleshoot missing windows event logs?

dillardo_2 — Fri, 06 Sep 2019 13:39:59 GMT

Have you checked your buckets? If they are being frozen prematurely due to storage pressure, the data might have been deleted before retention policy expiry.

Re: How do you troubleshoot missing windows event logs?

jeremyfer — Mon, 10 Aug 2020 00:11:02 GMT

Did you ever find a solution to this @ABasit10 ?
We have the exact same forwarder config on hundreds of other machines, and it's functioning fine. On one machine Splunk seems to be skipping lots of events, I'm looking specifically for 7001 (login), and its never there, but also even long after a login many events are missing. I can see in event viewer events A,B,C we might only get event B into Splunk.

E.g. Events in Event Viewer, only the highlighted ones are coming through.

But we seem to be missing a large selection of Events.

Related Forwarder Config

[WinEventLog://System] disabled = 0 start_from = oldest current_only = 0 checkpointInterval = 10 index = wineventlog renderXml=false

Re: How do you troubleshoot missing windows event logs?

to4kawa — Mon, 10 Aug 2020 00:16:00 GMT

https://qiita.com/odorusatoshi/items/5a703b9befc253ab7deb

@jeremyfer

this blog is japanese, but there is a lot of trouble shoot method.
please check this.