Duplicate Extracted Fields (ingest through HEC)

[mrstrozy]

Hi, I am seeing duplicate extractions for events in my Splunk instance. To give a background, I have a couple forwarders (which are mostly not used), an indexer cluster in which each indexer is running the Splunk HEC and the HEC is sitting behind a load balancer, I have one index cluster master and finally I have a search head cluster.

There is not a props.conf file on the search heads.

Currently the props.conf lives in the /opt/splunk/etc/system/local/ directory on the indexers. Here is a sample config:

[mycustomsourcetype]
DATETIME_CONFIG =
INDEXED_EXTRACTIONS = json
KV_MODE = none
AUTO_KV_JSON = false
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
TZ = UTC
category = Structured
description = ticket stuff
disabled = false
TRUNCATE = 0
pulldown_type = true

Considering the only props.conf that I believe should be dictating extractions is on the indexers, I'm very confused about how to fix this.

Any help would be greatly appreciated. Thanks!

[woodcock]

Set this on your Search Head:

props.conf

[mycustomsourcetype]
KV_MODE = none
AUTO_KV_JSON = false

[ruman_splunk]

I needed both KV_MODE and AUTO_KV_JSON, thanks @woodcock!

[harsmarvania57]

Hi,

Please configure below props.conf on Search Head.

[mycustomsourcetype]
KV_MODE = none

[mrstrozy]

The part I forgot to mention was that the events ingested prior to the architecture change did not have duplicate extractions which added to the confusion. I noticed that the replication and search factors were not being met and so I explicitly added to the master server.conf clustering section with "replication_factor = 3" (though I thought this was the case by default). After doing so the replication and search factors were then met and all ingested events from that point on did not have duplicate field extractions.

I am pretty confused by this so anyone with more advanced knowledge please feel free to chime in. I would love to fully understand why this was the case.