Getting Data In

duplicate JSON are coming in search

kirrusk
Communicator

I have a json file like below

{"env":"UAT","label":"jenkins-17887.api.v2.dm.btc","App":"dm-d-services","rlmtemplate":"f2_api_fed","lastupdate":2020-11-23 11:09:78:455,"region":"APAC"}{"env":"UAT","label":"jenkins-17687.api.v2.dm.btc","App":"dt-s-services","rlmtemplate":"f3_api_fed","lastupdate":2020-11-23 11:025:79:475,"region":"APAC"}{"env":"UAT","label":"jenkins-18657.api.v2.dm.btc","App":"dt-s-services","rlmtemplate":"f3_api_fed","lastupdate":2020-11-23 11:025:79:475,"region":"APAC"}{"env":"UAT","label":"jenkins-17637.api.v2.dm.btc","App":"dt-s-services","rlmtemplate":"f3_api_fed","lastupdate":2020-11-23 11:025:79:475,"region":"APAC"}

in splunk,

_raw contains valid json data for all events.

issue is all fields are multi valued containing two copies of the json object.

forwarding json data to splunk and props is installed on indexer not in heavy forwarder,

props.conf

[test_json]

INDEXED_EXTRACTIONS = JSON

KV_MODE = none

AUTO_KV_JSON = false

SHOULD_LINEMERGE = false

FYI , using splunk 8.0

tried by setting KV_MODE = JSON  but it is also not working

 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Are you perhaps using Verbose Mode?

The INDEXED_EXTRACTIONS=json setting tells Splunk to extract fields from each event at index time.  A Verbose Mode search then automatically extracts fields at search time.  The result is duplicates.

---
If this reply helps you, Karma would be appreciated.
0 Karma

kirrusk
Communicator

@richgalloway  Can u pls suggest how can i rectify this, it is running in verbose mode. I changed to fast mode and smart mode but duplicates are still coming

0 Karma

richgalloway
SplunkTrust
SplunkTrust

In splunk it's showing one event but while applying query on it , like query | table fields

duplicate values are coming in fields


How is Splunk showing one event without a query?  What is the query you use that is showing duplicate values?

---
If this reply helps you, Karma would be appreciated.
0 Karma

kirrusk
Communicator

@richgalloway  updated question

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...