Getting Data In

My Windows Event Log size increased dramatically after being forwarded to Splunk. Is there a way to reduce the volume of data received by Splunk?

cowlthrpdx
New Member

I configured a Windows 2012 server to forward AD event logs to Splunk, everything is working well except for the volume of data being sent to Splunk.

In ~24hr the Windows event log being forwarded increased by ~4GB on disk. The Splunk admin reported that in the same time period the Splunk server received ~70GB from the Windows server.

Other than filtering out events, is there a way to reduce the volume of data received by Splunk?

Thanks,
Rob

0 Karma

jonmargulies
Path Finder

This Splunk blog post has a trick that's fully supported and will reduce the size of your Windows events: https://www.splunk.com/blog/2014/11/04/splunk-6-2-feature-overview-xml-event-logs/. The post explains how to send your Windows logs in as XML, which is parsed perfectly by the Windows TA, but the events are a lot smaller because they have less duplicate data (in fact, the XML versions of the events have more useful information stuffed into fewer characters). According to the blog, it reduces event size by about 70%, but in practice it looked like a smaller reduction in my environment.

0 Karma

kkkelvinkk
New Member

Interesting. However, the following links seems report some different results.
https://www.batchworks.de/why-using-xml-event-logs-sucks-using-splunk/

I will give a try tommorow.

0 Karma

gfreitas
Builder

Is that a new server? Maybe you can add ignoreOlderThan = 24h on inputs.conf to make sure this data is really from today.

0 Karma
Get Updates on the Splunk Community!

Exporting Splunk Apps

Join us on Monday, October 21 at 11 am PT | 2 pm ET!With the app export functionality, app developers and ...

Cisco Use Cases, ITSI Best Practices, and More New Articles from Splunk Lantern

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Build Your First SPL2 App!

Watch the recording now!.Do you want to SPL™, too? SPL2, Splunk's next-generation data search and preparation ...