Getting Data In
Highlighted

How can I keep my data unique with manually uploading two overlapping log files?

New Member

How can manual data uploads with overlapping log files include only unique data? The goal is to avoid uploading duplicate data. For example, if log file #1 with data from Jan 1 - Jan 31 is uploaded and later log file #2 with data from Jan 15 - Feb 15 is uploaded, how can the duplicate data (Jan 15-31) in file #2 automatically not be uploaded? Note that we have a unique requirement that a log file is air gapped from any network, which does not allow us to use a Splunk forwarder.

Tags (2)
0 Karma
Highlighted

Re: How can I keep my data unique with manually uploading two overlapping log files?

SplunkTrust
SplunkTrust

Splunk doesn't deduplicate on upload. Different log files are assumed to have different data even if the times overlap.
Even if you were allowed to use one, the forwarder wouldn't help.

---
If this reply helps you, an upvote would be appreciated.
0 Karma