Building for the Splunk Platform

How to avoid splunk from indexing duplicate data? (Sending using python + HTML)

LAcioffi
Explorer

Hello everyone!

I am using a python script to send some data to a splunk instance on my own machine. The code is something like this:

service = client.connect(host='localhost',port=8089,username='SOMETHING',password='SOMETHING2')
myindex = service.indexes["indexName"]
myindex.submit(jsonData, sourcetype="bobby", host="local")

 

Is there any way to "tell" Splunk not to index anything duplicated? That is, anything that already exists in the index.

I know i could mess with the script to avoid sending duplicates, but if splunk could do it, things would be easier.

Thanks! 

Labels (1)
Tags (3)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust
Splunk does not and can not filter duplicates on ingest.
---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust
Splunk does not and can not filter duplicates on ingest.
---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...

Ready, Set, SOAR: How Utility Apps Can Up Level Your Playbooks!

 WATCH NOW Powering your capabilities has never been so easy with ready-made Splunk® SOAR Utility Apps. Parse ...