Hi @krutika_ag ... what @richgalloway said was an excellent answer. For Splunk newbies, let me rephrase it(the url link for your ref - https://docs.splunk.com/Documentation/Splunk/9.1.2/Data/Monitorfilesanddirectories) as follows: How the forwarder monitors archive files In order to monitor archived files, forwarders decompress archive files, such as a TAR or ZIP file, prior to processing. Splunk then processes these files in a "single threaded format" (there are pros and cons, but that is a different topic). The following types of archive files are supported: TAR GZ BZ2 TAR.GZ and TGZ TBZ and TBZ2 ZIP Z If you add new data to an existing archive file, the forwarder reprocesses the entire file rather than just the new data. This can result in event duplication. so, to avoid duplication, you should monitor the whole archive file. Lets say if these files are small, then you can monitor the whole archive and the license usage may not be impacted so much (the search time vs index time... should be considered clearly and well planned for this task). One more thing to consider: are you using UF or HF --- or both ---- or neither(you may directly upload thru SH GUI) - Splunk Support does not support this deployment model) hope this helped some new Splunkers, thanks.
... View more