Getting Data In

How can I keep my data unique with manually uploading two overlapping log files?

mkawamura
New Member

How can manual data uploads with overlapping log files include only unique data? The goal is to avoid uploading duplicate data. For example, if log file #1 with data from Jan 1 - Jan 31 is uploaded and later log file #2 with data from Jan 15 - Feb 15 is uploaded, how can the duplicate data (Jan 15-31) in file #2 automatically not be uploaded? Note that we have a unique requirement that a log file is air gapped from any network, which does not allow us to use a Splunk forwarder.

Tags (2)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Splunk doesn't deduplicate on upload. Different log files are assumed to have different data even if the times overlap.
Even if you were allowed to use one, the forwarder wouldn't help.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...