Getting Data In
Highlighted

REST endpoint: data/indexes-extended - Why is total_raw_size is bigger than total_size

Explorer

I tried to interpret the output the REST endpoint from Splunk doc:
http://docs.splunk.com/Documentation/Splunk/7.0.2/RESTREF/RESTintrospect#data.2Findexes-extended.2F....
and have problem understanding the 2 output parameters totalrawsize and total_size

API:
data/indexes-extended/{name}

Usage details
totalrawsize (If totalsize > 0) Cumulative size (fractional MB) on disk of the /rawdata/ directories of all buckets in this index, excluding frozen.
total
size Size (fractional MB) on disk of this index.

Example:
28.000/s:key
22.000/s:key

Question:
Why is totalrawsize bigger the total_size? Note that I got the same result when applying this API on my cluster.

0 Karma
Highlighted

Re: REST endpoint: data/indexes-extended - Why is total_raw_size is bigger than total_size

Influencer

Hi,

rawSize: The volume in bytes of the raw data files in each bucket. This value represents the volume before compression and the addition of index files.

sizeOnDisk: The size in MB of disk space that the bucket takes up expressed as a floating point number. This value represents the volume of the compressed raw data files and the index files.

http://docs.splunk.com/Documentation/Splunk/7.0.2/SearchReference/Dbinspect

Thanks
Strive

Highlighted

Re: REST endpoint: data/indexes-extended - Why is total_raw_size is bigger than total_size

Motivator

totalrawsize: essentially uncompressed bytes indexed on this indexer for this index
total_size: essentially size on disk for after compression and indexing metadata on this indexer for this index

On average it will be normal for totalsize to be 50% of totalraw_size.

0 Karma