Getting Data In

Using INDEXED_EXTRACTIONS=json produces duplicate values

mgallacher
Engager

Before you ask, I have found at least 10 questions similar to this as well as two identical questions, both of which are unresolved.

I have one sourcetype which extracts fields from a JSON properly. Awesome, no problem. I created a second sourcetype with the same settings and all fields are extracted twice during a search. The only difference in the data is the first sourcetype has the JSON on a single line. The second sourcetype has the JSON indented on multiple lines. This results in a multi-value field (not a duplicate event.)

I'm running v7.0.1 with forwarders. I am a loss of what to even check next. Suggestions???

Thanks!

FIRST (ORIGINAL-WORKS FINE)

SHOULD_LINEMERGE = true
INDEXED_EXTRACTIONS = json
NO_BINARY_CHECK = true
CHARSET=UTF-8
KV_MODE = none
AUTO_KV_JSON = false
category=Structured
description=JavaScript Object...
disabled=false
pulldown_type=true
TIMESTAMP_FIELDS = timestamp
TIME_FORMAT=%Y-%m-%dT%H%M%S%Z
TRUNCATE=0

SECOND (EXTRACTS DUPLICATES)

INDEXED_EXTRACTIONS = json
NO_BINARY_CHECK = true
CHARSET=UTF-8
KV_MODE = none
AUTO_KV_JSON = false
category=Structured
description=JavaScript Object...
disabled=false
pulldown_type=true
TIMESTAMP_FIELDS = timestamp
TIME_FORMAT=%Y-%m-%dT%H%M%S%Z
TRUNCATE=0

along with all combinations of

BREAK_ONLY_BEFORE_DATE = [true | false]
SHOULD_LINEMERGE = [true | false]

mstrozyk
Engager

Having the same exact problem and I can't figure it out.

0 Karma

iparitosh
Path Finder

What’s the name of your two sourectypes ?
Where have you deployed them? SH or forwarder?

0 Karma
Get Updates on the Splunk Community!

Splunk ITSI & Correlated Network Visibility

  Now On Demand   Take Your Network Visibility to the Next Level In today’s complex IT environments, ...

Leveraging Detections from the Splunk Threat Research Team & Cisco Talos

  Now On Demand  Stay ahead of today’s evolving threats with the combined power of the Splunk Threat Research ...

New in Splunk Observability Cloud: Automated Archiving for Unused Metrics

Automated Archival is a new capability within Metrics Management; which is a robust usage & cost optimization ...