Indexing thousands of zip files in Splunk

[ips_mandar]

I have one folder where everyday thousands of zip files were added and I want to monitor this folder via Splunk.
So What is best practice to monitor these zip files in splunk and will it impact performance if I monitor zip files directly or shall I extract these file first and then monitor?
Please guide me..

[hunderliggur]

It depends by what you mean by monitor and what your resources are. If you are using a Universal Forwarder on the system to monitor the folder I would just configure the UF to input the files as is (zipped).

You do not say if your Splunk Enterprise deployment is a single node (Search+Index on a box) or distributed with seperate search, index, forward nodes. I would first try just monitoring the zip as is and make sure you are not overloading the node doing the input. That would be the simplest/cleanest approach and simplify your operations.

[esix_splunk]

Do note that Splunk's unzip process is single threaded. So with 1000s of zip files, you're looking at quite a time to unzip and ingest. You'd be better to unzip the files and use a sinkhole monitor to ingest these file. Much more efficient and will get them indexed much quicker.

[arunsundarm]

Splunk will be able to handle zip files, however you can extract them and then ingest the data

[arunsundarm]

Just simple mention the source where the zip files needs to be ingested and splunk should decompress and ingest them