Installation

Why am I seeing a difference between size reported by eventcount and size reported by _internal index?

wkupersa
Path Finder

I created an index today and started feeding data to it. I ran two different searches to get the size of the index, and I get two very different answers. Can someone help me to understand why?

index="_internal" source="*metrics.log" per_index_thruput series="willie" | stats sum(kb) AS kbytes| eval bytes = kbytes * 1024

bytes = 33,331,904.006

| eventcount summarize=false report_size=true index=willie | stats sum(size_bytes) AS bytes

bytes = 4,740,441

Labels (1)
0 Karma
1 Solution

esix_splunk
Splunk Employee
Splunk Employee

What you are seeing from internal is actual amount of data being sent to splunk for the Series willie before it is compressed and reduced.

When you do the eventcount, that pulls the information for the indexed data on disk. So you can see that you sent ~33mb of data. Then the eventcount for the data is ~5mg. So this shows the compression that has been done on the index also. This is strictly event data, and doesnt include metadata or tsidx for the index though.

View solution in original post

esix_splunk
Splunk Employee
Splunk Employee

What you are seeing from internal is actual amount of data being sent to splunk for the Series willie before it is compressed and reduced.

When you do the eventcount, that pulls the information for the indexed data on disk. So you can see that you sent ~33mb of data. Then the eventcount for the data is ~5mg. So this shows the compression that has been done on the index also. This is strictly event data, and doesnt include metadata or tsidx for the index though.

wkupersa
Path Finder

So would thins indicate that _internal is a decent indicator of license usage for the data, while eventcount (plus a bit for metadata, etc. ) is a good indicator of storage use? Thanks for the answer!

0 Karma
Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...