Getting Data In

REST endpoint: data/indexes-extended - Why is total_raw_size is bigger than total_size

thenhaque
Explorer

I tried to interpret the output the REST endpoint from Splunk doc:
http://docs.splunk.com/Documentation/Splunk/7.0.2/RESTREF/RESTintrospect#data.2Findexes-extended.2F....
and have problem understanding the 2 output parameters total_raw_size and total_size

API:
data/indexes-extended/{name}

Usage details
total_raw_size (If total_size > 0) Cumulative size (fractional MB) on disk of the /rawdata/ directories of all buckets in this index, excluding frozen.
total_size Size (fractional MB) on disk of this index.

Example:
28.000/s:key
22.000/s:key

Question:
Why is total_raw_size bigger the total_size? Note that I got the same result when applying this API on my cluster.

0 Karma

bandit
Motivator

total_raw_size: essentially uncompressed bytes indexed on this indexer for this index
total_size: essentially size on disk for after compression and indexing metadata on this indexer for this index

On average it will be normal for total_size to be 50% of total_raw_size.

0 Karma

strive
Influencer

Hi,

rawSize: The volume in bytes of the raw data files in each bucket. This value represents the volume before compression and the addition of index files.

sizeOnDisk: The size in MB of disk space that the bucket takes up expressed as a floating point number. This value represents the volume of the compressed raw data files and the index files.

http://docs.splunk.com/Documentation/Splunk/7.0.2/SearchReference/Dbinspect

Thanks
Strive

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Persistent Queue at TcpOut — One of Splunk's Most Practical Features

Splunk introduced persistent queueing at the tcpout layer as one of the most practical resilience features in ...

Skip the Awkward Silence: Have a .conf-ersation at .conf26

Picture this. You arrive at .conf26 already having your socializing and networking plans mapped out. No ...

Rethinking Zero Trust: From Product Purchases to Logical Control Evidence

Implementing Zero Trust (ZT) across complex environments often falters at the very beginning due to a ...