Getting Data In

duplicate JSON are coming in search

kirrusk
Communicator

I have a json file like below

{"env":"UAT","label":"jenkins-17887.api.v2.dm.btc","App":"dm-d-services","rlmtemplate":"f2_api_fed","lastupdate":2020-11-23 11:09:78:455,"region":"APAC"}{"env":"UAT","label":"jenkins-17687.api.v2.dm.btc","App":"dt-s-services","rlmtemplate":"f3_api_fed","lastupdate":2020-11-23 11:025:79:475,"region":"APAC"}{"env":"UAT","label":"jenkins-18657.api.v2.dm.btc","App":"dt-s-services","rlmtemplate":"f3_api_fed","lastupdate":2020-11-23 11:025:79:475,"region":"APAC"}{"env":"UAT","label":"jenkins-17637.api.v2.dm.btc","App":"dt-s-services","rlmtemplate":"f3_api_fed","lastupdate":2020-11-23 11:025:79:475,"region":"APAC"}

in splunk,

_raw contains valid json data for all events.

issue is all fields are multi valued containing two copies of the json object.

forwarding json data to splunk and props is installed on indexer not in heavy forwarder,

props.conf

[test_json]

INDEXED_EXTRACTIONS = JSON

KV_MODE = none

AUTO_KV_JSON = false

SHOULD_LINEMERGE = false

FYI , using splunk 8.0

tried by setting KV_MODE = JSON  but it is also not working

 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Are you perhaps using Verbose Mode?

The INDEXED_EXTRACTIONS=json setting tells Splunk to extract fields from each event at index time.  A Verbose Mode search then automatically extracts fields at search time.  The result is duplicates.

---
If this reply helps you, Karma would be appreciated.
0 Karma

kirrusk
Communicator

@richgalloway  Can u pls suggest how can i rectify this, it is running in verbose mode. I changed to fast mode and smart mode but duplicates are still coming

0 Karma

richgalloway
SplunkTrust
SplunkTrust

In splunk it's showing one event but while applying query on it , like query | table fields

duplicate values are coming in fields


How is Splunk showing one event without a query?  What is the query you use that is showing duplicate values?

---
If this reply helps you, Karma would be appreciated.
0 Karma

kirrusk
Communicator

@richgalloway  updated question

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Modernize your Splunk Apps – Introducing Python 3.13 in Splunk

We are excited to announce that the upcoming releases of Splunk Enterprise 10.2.x and Splunk Cloud Platform ...

Step into “Hunt the Insider: An Splunk ES Premier Mystery” to catch a cybercriminal ...

After a whole week of being on call, you fell asleep on your keyboard, and you hit a sequence of buttons that ...