Getting Data In

load compressed files

dmlee
Communicator

Hi,

as we know , before splunk eat a compressed file, splunk will decompress it first then index it.

but, if we have many compressed files under the same directory (ex: ap_20110301.zip, ap_20110302.zip ...) and their original file name are the same (ex:ap.log), what will happen ?

will splunk decompress all those files then index them ? or decompress and index one by one ?

because their original file name are the same , if splunk decompress all of the files at first , it will overwrite existing files (actually, this is what we observed, but we want to make sure).

thanks.

Tags (1)
1 Solution

Stephen_Sorkin
Splunk Employee
Splunk Employee

Splunk never actually decompresses the files within archives to a temporary location on disk. Instead we use a library called "libarchive" that allows us to stream through the contents of archives. These streamed contents are then indexed.

View solution in original post

Stephen_Sorkin
Splunk Employee
Splunk Employee

Splunk never actually decompresses the files within archives to a temporary location on disk. Instead we use a library called "libarchive" that allows us to stream through the contents of archives. These streamed contents are then indexed.

dmlee
Communicator

lessons learned, thanks

0 Karma
Get Updates on the Splunk Community!

Now Playing: Splunk Education Summer Learning Premieres

It’s premiere season, and Splunk Education is rolling out new releases you won’t want to miss. Whether you’re ...

The Visibility Gap: Hybrid Networks and IT Services

The most forward thinking enterprises among us see their network as much more than infrastructure – it's their ...

Get Operational Insights Quickly with Natural Language on the Splunk Platform

In today’s fast-paced digital world, turning data into actionable insights is essential for success. With ...