Getting Data In

Using INDEXED_EXTRACTIONS=json produces duplicate values

mgallacher
Engager

Before you ask, I have found at least 10 questions similar to this as well as two identical questions, both of which are unresolved.

I have one sourcetype which extracts fields from a JSON properly. Awesome, no problem. I created a second sourcetype with the same settings and all fields are extracted twice during a search. The only difference in the data is the first sourcetype has the JSON on a single line. The second sourcetype has the JSON indented on multiple lines. This results in a multi-value field (not a duplicate event.)

I'm running v7.0.1 with forwarders. I am a loss of what to even check next. Suggestions???

Thanks!

FIRST (ORIGINAL-WORKS FINE)

SHOULD_LINEMERGE = true
INDEXED_EXTRACTIONS = json
NO_BINARY_CHECK = true
CHARSET=UTF-8
KV_MODE = none
AUTO_KV_JSON = false
category=Structured
description=JavaScript Object...
disabled=false
pulldown_type=true
TIMESTAMP_FIELDS = timestamp
TIME_FORMAT=%Y-%m-%dT%H%M%S%Z
TRUNCATE=0

SECOND (EXTRACTS DUPLICATES)

INDEXED_EXTRACTIONS = json
NO_BINARY_CHECK = true
CHARSET=UTF-8
KV_MODE = none
AUTO_KV_JSON = false
category=Structured
description=JavaScript Object...
disabled=false
pulldown_type=true
TIMESTAMP_FIELDS = timestamp
TIME_FORMAT=%Y-%m-%dT%H%M%S%Z
TRUNCATE=0

along with all combinations of

BREAK_ONLY_BEFORE_DATE = [true | false]
SHOULD_LINEMERGE = [true | false]

mstrozyk
Engager

Having the same exact problem and I can't figure it out.

0 Karma

iparitosh
Path Finder

What’s the name of your two sourectypes ?
Where have you deployed them? SH or forwarder?

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...