Installation

Why am I seeing a difference between size reported by eventcount and size reported by _internal index?

wkupersa
Path Finder

I created an index today and started feeding data to it. I ran two different searches to get the size of the index, and I get two very different answers. Can someone help me to understand why?

index="_internal" source="*metrics.log" per_index_thruput series="willie" | stats sum(kb) AS kbytes| eval bytes = kbytes * 1024

bytes = 33,331,904.006

| eventcount summarize=false report_size=true index=willie | stats sum(size_bytes) AS bytes

bytes = 4,740,441

Labels (1)
0 Karma
1 Solution

esix_splunk
Splunk Employee
Splunk Employee

What you are seeing from internal is actual amount of data being sent to splunk for the Series willie before it is compressed and reduced.

When you do the eventcount, that pulls the information for the indexed data on disk. So you can see that you sent ~33mb of data. Then the eventcount for the data is ~5mg. So this shows the compression that has been done on the index also. This is strictly event data, and doesnt include metadata or tsidx for the index though.

View solution in original post

esix_splunk
Splunk Employee
Splunk Employee

What you are seeing from internal is actual amount of data being sent to splunk for the Series willie before it is compressed and reduced.

When you do the eventcount, that pulls the information for the indexed data on disk. So you can see that you sent ~33mb of data. Then the eventcount for the data is ~5mg. So this shows the compression that has been done on the index also. This is strictly event data, and doesnt include metadata or tsidx for the index though.

wkupersa
Path Finder

So would thins indicate that _internal is a decent indicator of license usage for the data, while eventcount (plus a bit for metadata, etc. ) is a good indicator of storage use? Thanks for the answer!

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...