Getting Data In

What is the size (in bytes) of various common IT data event sourcetypes

maverick
Splunk Employee
Splunk Employee

I was wondering if anyone had a link to some web page that lists the sizes (in bytes) for various common IT data event source types, like Cisco ASA, Microsoft IIS, Bluecoat, WebSphere/WebLogic log4j or logback, insert_your_common_sourcetype_here, etc.

maverick
Splunk Employee
Splunk Employee

Please see this Splunk Wiki table for more details, or to add your own events and their sizes now:

http://www.splunk.com/wiki/Community:CommonEventSizes

mendesjo
Path Finder

Any idea how you would find the TOTAL size of events by sourcetype in an index?

0 Karma

maverick
Splunk Employee
Splunk Employee

Thanks! This will help a lot!

0 Karma

hexx
Splunk Employee
Splunk Employee

Here's the same search but also showing the 10th and 90th percentile for event size (in bytes) broken down by sourcetype :

  • | eval esize=len(_raw) | stats p10(esize), avg(esize), p90(esize) by sourcetype
0 Karma

hexx
Splunk Employee
Splunk Employee

If you want to check the average size in bytes of your events broken down by sourcetype, you can run the search below. Of course, feel free to replace "*" with a specific data set you want to study, and don't forget to adequately set the time frame of the search :

  • | eval esize=len(_raw) | stats avg(esize) by sourcetype
0 Karma

effem
Communicator

Isn't it simply the length of the _raw field? e.g. the value given by esize is only the number of characters.

0 Karma
Get Updates on the Splunk Community!

Infographic provides the TL;DR for the 2024 Splunk Career Impact Report

We’ve been buzzing with excitement about the recent validation of Splunk Education! The 2024 Splunk Career ...

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...