Installation

Why am I seeing a difference between size reported by eventcount and size reported by _internal index?

wkupersa
Path Finder

I created an index today and started feeding data to it. I ran two different searches to get the size of the index, and I get two very different answers. Can someone help me to understand why?

index="_internal" source="*metrics.log" per_index_thruput series="willie" | stats sum(kb) AS kbytes| eval bytes = kbytes * 1024

bytes = 33,331,904.006

| eventcount summarize=false report_size=true index=willie | stats sum(size_bytes) AS bytes

bytes = 4,740,441

Labels (1)
0 Karma
1 Solution

esix_splunk
Splunk Employee
Splunk Employee

What you are seeing from internal is actual amount of data being sent to splunk for the Series willie before it is compressed and reduced.

When you do the eventcount, that pulls the information for the indexed data on disk. So you can see that you sent ~33mb of data. Then the eventcount for the data is ~5mg. So this shows the compression that has been done on the index also. This is strictly event data, and doesnt include metadata or tsidx for the index though.

View solution in original post

esix_splunk
Splunk Employee
Splunk Employee

What you are seeing from internal is actual amount of data being sent to splunk for the Series willie before it is compressed and reduced.

When you do the eventcount, that pulls the information for the indexed data on disk. So you can see that you sent ~33mb of data. Then the eventcount for the data is ~5mg. So this shows the compression that has been done on the index also. This is strictly event data, and doesnt include metadata or tsidx for the index though.

wkupersa
Path Finder

So would thins indicate that _internal is a decent indicator of license usage for the data, while eventcount (plus a bit for metadata, etc. ) is a good indicator of storage use? Thanks for the answer!

0 Karma
Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...