Getting Data In

Why is JSON Event Data Showing Duplicate Values?

cchange
Path Finder

Hi,

I'm trying to ingest json data but it showing data twice for each event field. I used below in props.conf and not sure what is causing the issue.

[sourcetype]

INDEXED_EXTRACTIONS = json

KV_MODE=none  #( tried both KV_MODE=json and None)

SHOULD_LINEMERGE=true

pulldown_type=true

TIMESTAMP_FIELDS=<timestamp field>

AUTO_KV_JSON=false

NO_BINARY_CHECK = true

 

Data in SH:

Title:  [RESOLVED] Increased Error Rates 

            [RESOLVED] Increased Error Rates

 

Labels (1)
0 Karma

marycordova
SplunkTrust
SplunkTrust

it's the "INDEXED_EXTRACTIONS"

Splunk is doing search time extractions natively on the JSON data anyway and the indexed extractions are adding the duplicate, as well as using up more disk space, you can remove that config and everything should be fine

pretty sure you could just drop the entire KV_MODE and AUTO_KV_JSON lines as well and search time parsing would still extract all your fields for you

here's an exact config I just setup the other day:

[batch:///opt/logs/*.json.gz]
disabled = false
index = <index>
sourcetype = <sourcetype>
move_policy = sinkhole


[sourcetype]
disabled=false
NO_BINARY_CHECK = true
LINE_BREAKER = <regex>
SHOULD_LINEMERGE = false
TRUNCATE = 0
DATETIME_CONFIG = <time>

 

-upvotes appreciated 🤓

@marycordova

jesspetty
Engager

this worked for me

0 Karma

cchange
Path Finder

Thanks for the response but it is not working.

0 Karma

diogofgm
SplunkTrust
SplunkTrust

Hi,
The "Data in SH:" doesn't show much. It's linking to you Splunk local instance. Can you correct that and share an example of what you're seeing?

------------
Hope I was able to help you. If so, some karma would be appreciated.
0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Can’t Make It to Boston? Stream .conf25 and Learn with Haya Husain

Boston may be buzzing this September with Splunk University and .conf25, but you don’t have to pack a bag to ...

Splunk Lantern’s Guide to The Most Popular .conf25 Sessions

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Unlock What’s Next: The Splunk Cloud Platform at .conf25

In just a few days, Boston will be buzzing as the Splunk team and thousands of community members come together ...