HI all ,
I want o index specific files inside a zip folder. Directory structure is like this.
D:/Reports/abc.zip/x(empty folder)/y(empty folder)/z(empty folder)/abc.txt,pqr.csv,lmn.csv
Also the tool will generate same zip structure i. D:/reports but the zip file name will be different. Rest all the structure beneath the .zip will be same like below
D:/Reports/abc.zip/x(empty folder)/y(empty folder)/z(empty folder)/abc.txt,pqr.csv,lmn.csv
D:/Reports/123.zip/x(empty folder)/y(empty folder)/z(empty folder)/abc.txt,pqr.csv,lmn.csv
I have 2 questions
PLs help !!
You can't index only a particular file within a zip file. I would actually suggest that you not index the data while it's contained within a zip file because the ArchiveProcessor of splunkd is single threaded and can only read one file at a time. This can be a significant bottleneck in terms of performance on the Splunk instance reading this data. The MUCH better alternative would be to unpack the files with a script and monitor the folder that contains the unpacked files. While it might be easier just to monitor the zip files, it really isn't worth the trouble the bottleneck can cause.
If you've got no alternative but to monitor the zip files, then what you can do is to throw away any data that doesn't match a pattern specified using props and transforms by sending it to nullQueue You can find instructions for that here.