How does the index point to compressed raw files?

[kgregor]

Hey there,

quick question: the raw data in the buckets are stored in a compressed way. The index files point to these raw data files. Now, when I search for events, I assume that Splunk goes through the index, but then needs to return the original events to me. Obviously, when the result shows up, it is in plain text. So when and how does this decompression of the raw data take place?

Thanks and best regards

Konstantin

[hunters_splunk]

Hi kgregor,

You are right - post-indexing compression occurs when Splunk writes the raw data and index files to disk.
When you search indexed data, Splunk retrieves data from the _raw field, which contains the original raw data of an event. The search command uses the data in _raw when performing searches and data extraction. I think this is where decompression occurs.
For more detailed information about how splunk handles data, please refer to this topic:
http://docs.splunk.com/Documentation/Splunk/6.5.0/Data/WhatSplunkdoeswithyourdata

Hope it helps. Thanks!
Hunter Shen