https://community.splunk.com/t5/Getting-Data-In/Are-there-any-issues-with-Splunk-reading-and-indexing-gzip-files/m-p/189581#M37766 <P>That all sounds reasonable as long as it reliable here. These are daily batch files, no manageable delay is really a problem, and it's done overnight when things are relatively sleepy. Where would the files be decompressed to by default?</P> <P>Ultimately this is a temp hack before we get a real time stream of equivalent data, so looks good all round to me. Thanks</P> Thu, 19 Mar 2015 13:52:57 GMT acidkewpie 2015-03-19T13:52:57Z https://community.splunk.com/t5/Getting-Data-In/Are-there-any-issues-with-Splunk-reading-and-indexing-gzip-files/m-p/189578#M37763 <P>Hi, </P> <P>I've heard comments against configuring Splunk to read gzipped files, horror stories of it not always noticing the file was indeed a gz and logging the compressed raw data instead. I'm looking to piggyback on an existing process that drops a pile of gzipped logs onto a server with a universal forwarder already installed, and don't want to have to delve into custom scripts to first decompress the files to a temp location if there are no genuine known concerns around Splunk's consistent reliability when it comes to indexing gzipped files..</P> Thu, 19 Mar 2015 12:13:58 GMT https://community.splunk.com/t5/Getting-Data-In/Are-there-any-issues-with-Splunk-reading-and-indexing-gzip-files/m-p/189578#M37763 acidkewpie 2015-03-19T12:13:58Z https://community.splunk.com/t5/Getting-Data-In/Are-there-any-issues-with-Splunk-reading-and-indexing-gzip-files/m-p/189579#M37764 <P>For my understand there is no need to decompress gzip files before indexing it.</P> Thu, 19 Mar 2015 13:19:42 GMT https://community.splunk.com/t5/Getting-Data-In/Are-there-any-issues-with-Splunk-reading-and-indexing-gzip-files/m-p/189579#M37764 btt 2015-03-19T13:19:42Z https://community.splunk.com/t5/Getting-Data-In/Are-there-any-issues-with-Splunk-reading-and-indexing-gzip-files/m-p/189580#M37765 <P>Splunk can read zip/gzip files. Do understand that what Splunk does on the back end is:</P> <P>1) Unarchives <BR /> 2) Reads the Files<BR /> 3) Indexes<BR /> 4) Deletes the unarchived pieces</P> <P>Additionally, the unzip process is not multithreaded. So you can see a fair amount of latency and cpu time used when this is done. Especially true if you are trying to monitor a large number of zip files. Also, you have to becareful regarding free disk space..</P> Thu, 19 Mar 2015 13:33:40 GMT https://community.splunk.com/t5/Getting-Data-In/Are-there-any-issues-with-Splunk-reading-and-indexing-gzip-files/m-p/189580#M37765 esix_splunk 2015-03-19T13:33:40Z https://community.splunk.com/t5/Getting-Data-In/Are-there-any-issues-with-Splunk-reading-and-indexing-gzip-files/m-p/189581#M37766 <P>That all sounds reasonable as long as it reliable here. These are daily batch files, no manageable delay is really a problem, and it's done overnight when things are relatively sleepy. Where would the files be decompressed to by default?</P> <P>Ultimately this is a temp hack before we get a real time stream of equivalent data, so looks good all round to me. Thanks</P> Thu, 19 Mar 2015 13:52:57 GMT https://community.splunk.com/t5/Getting-Data-In/Are-there-any-issues-with-Splunk-reading-and-indexing-gzip-files/m-p/189581#M37766 acidkewpie 2015-03-19T13:52:57Z