Building for the Splunk Platform

How to avoid splunk from indexing duplicate data? (Sending using python + HTML)

LAcioffi
Explorer

Hello everyone!

I am using a python script to send some data to a splunk instance on my own machine. The code is something like this:

service = client.connect(host='localhost',port=8089,username='SOMETHING',password='SOMETHING2')
myindex = service.indexes["indexName"]
myindex.submit(jsonData, sourcetype="bobby", host="local")

 

Is there any way to "tell" Splunk not to index anything duplicated? That is, anything that already exists in the index.

I know i could mess with the script to avoid sending duplicates, but if splunk could do it, things would be easier.

Thanks! 

Labels (1)
Tags (3)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust
Splunk does not and can not filter duplicates on ingest.
---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust
Splunk does not and can not filter duplicates on ingest.
---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...

What's New in Splunk Cloud Platform 9.0.2208?!

Howdy!  We are happy to share the newest updates in Splunk Cloud Platform 9.0.2208! Analysts can benefit ...

Admin Console: A Single, Unified Interface for All Your Cloud Admin Needs

WATCH NOWJoin us to learn how the admin console can save you time and give you more control over the Splunk® ...