topic Why is JSON Event Data Showing Duplicate Values? in Getting Data In

Why is JSON Event Data Showing Duplicate Values?

cchange — Thu, 03 Mar 2022 00:29:37 GMT

Hi,

I'm trying to ingest json data but it showing data twice for each event field. I used below in props.conf and not sure what is causing the issue.

[sourcetype]

INDEXED_EXTRACTIONS = json

KV_MODE=none #( tried both KV_MODE=json and None)

SHOULD_LINEMERGE=true

pulldown_type=true

TIMESTAMP_FIELDS=<timestamp field>

AUTO_KV_JSON=false

NO_BINARY_CHECK = true

Data in SH:

Title: [RESOLVED] Increased Error Rates

[RESOLVED] Increased Error Rates

Re: JSON Event Data Showing Duplicate Values

diogofgm — Fri, 19 Jun 2020 01:01:34 GMT

Hi,
The "Data in SH:" doesn't show much. It's linking to you Splunk local instance. Can you correct that and share an example of what you're seeing?

Re: JSON Event Data Showing Duplicate Values

marycordova — Fri, 19 Jun 2020 01:43:34 GMT

it's the "INDEXED_EXTRACTIONS"

Splunk is doing search time extractions natively on the JSON data anyway and the indexed extractions are adding the duplicate, as well as using up more disk space, you can remove that config and everything should be fine

pretty sure you could just drop the entire KV_MODE and AUTO_KV_JSON lines as well and search time parsing would still extract all your fields for you

here's an exact config I just setup the other day:

[batch:///opt/logs/*.json.gz] disabled = false index = <index> sourcetype = <sourcetype> move_policy = sinkhole [sourcetype] disabled=false NO_BINARY_CHECK = true LINE_BREAKER = <regex> SHOULD_LINEMERGE = false TRUNCATE = 0 DATETIME_CONFIG = <time>

-upvotes appreciated 🤓

Re: JSON Event Data Showing Duplicate Values

cchange — Fri, 19 Jun 2020 19:50:55 GMT

Thanks for the response but it is not working.

Re: JSON Event Data Showing Duplicate Values

jesspetty — Thu, 03 Mar 2022 00:18:27 GMT

this worked for me