How to review Windows event logs to reduce noise?

[Taruchit]

Hi All,

Windows event logs generate large volumes of data every day.  Thus, there is excessive data ingestion making data noisy and difficult to analyze. 

I need your help to understand how to find the events which can be filtered out to reduce the volume of ingested data without losing visibility of important events which help to track security issues. 

Thank you

[Taruchit]

Hi @richgalloway and @gcusello,

Thank you for sharing your inputs.

index=xxx
|stats count BY EventCode

 I tried to fetch the current set of EventCodes ingested in the logs and some of the observations are: -

• There are close to 800 event codes ingested in the logs at present.
• Around 60% of the logs belong to single event code: 4663.
• Some of the other event codes with substantial number of logs: - 4672, 4688, 5449.

Thus, logs with Event Code: 4663 are relatively very high as compared to others. And thus, as a staring point I am looking to find the ways to analyze events with Event Code=4663, and keep the ones which are useful, and filter out the ones which do not provide details related to security issues. 

@gcusello : - I will try fetching logs based on sourcetype and see if logs which are verbose can be filtered out. 

I started off by fetching all values under field: - ProcessName, and computed Pareto Analysis to get a subset of those values which contribute most number of logs for the EventCode. Please do share if this can be an appropriate approach or if you see any issues with it.

In case there are any other suggestions, documents, previous .conf or usergroup sessions which talk about reviewing of logs and ways for ingesting good quality logs, please share.

Thank you

[gcusello]

Hi @Taruchit.,

as @richgalloway said, all data could be security relevant and if you filter something you could loose a relevant information.

But anyway I understand that you need to save license.

So at first enable only winEventLog:Security events, avoiding performance logs that are much verbose, then run a search like the following:

index=wineventlog
| stats count BY sourcetype

identidying the most verbose EventCode,

then if some of them aren't relevant for you security scopes you could blocklist them.

Ciao.

Giuseppe

[richgalloway]

Splunk says all data is security-relevant, but I understand wanting to ingest only the data most relevant to your security needs.  There is our problem - we don't know what security issues you want or need to track.  We also don't know what Windows events you're logging now.

I suggest logging only WinEventLog:Security events.  Furthermore, consider logging only the specific event codes used by your security use cases.