Getting Data In

duplicate JSON are coming in search

kirrusk
Communicator

I have a json file like below

{"env":"UAT","label":"jenkins-17887.api.v2.dm.btc","App":"dm-d-services","rlmtemplate":"f2_api_fed","lastupdate":2020-11-23 11:09:78:455,"region":"APAC"}{"env":"UAT","label":"jenkins-17687.api.v2.dm.btc","App":"dt-s-services","rlmtemplate":"f3_api_fed","lastupdate":2020-11-23 11:025:79:475,"region":"APAC"}{"env":"UAT","label":"jenkins-18657.api.v2.dm.btc","App":"dt-s-services","rlmtemplate":"f3_api_fed","lastupdate":2020-11-23 11:025:79:475,"region":"APAC"}{"env":"UAT","label":"jenkins-17637.api.v2.dm.btc","App":"dt-s-services","rlmtemplate":"f3_api_fed","lastupdate":2020-11-23 11:025:79:475,"region":"APAC"}

in splunk,

_raw contains valid json data for all events.

issue is all fields are multi valued containing two copies of the json object.

forwarding json data to splunk and props is installed on indexer not in heavy forwarder,

props.conf

[test_json]

INDEXED_EXTRACTIONS = JSON

KV_MODE = none

AUTO_KV_JSON = false

SHOULD_LINEMERGE = false

FYI , using splunk 8.0

tried by setting KV_MODE = JSON  but it is also not working

 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Are you perhaps using Verbose Mode?

The INDEXED_EXTRACTIONS=json setting tells Splunk to extract fields from each event at index time.  A Verbose Mode search then automatically extracts fields at search time.  The result is duplicates.

---
If this reply helps you, Karma would be appreciated.
0 Karma

kirrusk
Communicator

@richgalloway  Can u pls suggest how can i rectify this, it is running in verbose mode. I changed to fast mode and smart mode but duplicates are still coming

0 Karma

richgalloway
SplunkTrust
SplunkTrust

In splunk it's showing one event but while applying query on it , like query | table fields

duplicate values are coming in fields


How is Splunk showing one event without a query?  What is the query you use that is showing duplicate values?

---
If this reply helps you, Karma would be appreciated.
0 Karma

kirrusk
Communicator

@richgalloway  updated question

0 Karma
Get Updates on the Splunk Community!

CX Day is Coming!

Customer Experience (CX) Day is on October 7th!! We're so excited to bring back another day full of wonderful ...

Strengthen Your Future: A Look Back at Splunk 10 Innovations and .conf25 Highlights!

The Big One: Splunk 10 is Here!  The moment many of you have been waiting for has arrived! We are thrilled to ...

Now Offering the AI Assistant Usage Dashboard in Cloud Monitoring Console

Today, we’re excited to announce the release of a brand new AI assistant usage dashboard in Cloud Monitoring ...