Getting Data In

How to remove the duplicate values from json events

Nadhiyaa
Path Finder

alt text

Below is sample data . How to remove the duplicate values

Tags (1)
0 Karma

jawaharas
Motivator

Below configuration will help to remove duplicates in JSON events.

props.conf in Indexer

[<source_type>]
INDEXED_EXTRACTIONS = json
category = Structured

props.conf in Search head

[<source_type>]
AUTO_KV_JSON = false
KV_MODE = none

This answer is based on input from @harsmarvania57. Thanks.

jawaharas
Motivator

@Nadhiyaa
Kindly accept the answer if it helped you, so others can refer it.

0 Karma

harsmarvania57
Ultra Champion

Hi,

It looks like you are using INDEXED_EXTRACTIONS = json and KV_MODE = json. If you are using INDEXED_EXTRACTIONS = json while ingesting the data then set KV_MODE = none on Search Head and it will not display duplicate value.

DavidHourani
Super Champion

@Nadhiyaa, are you using stats or dedup ? You shouldnt have duplicated if that's the case. Could you please post your query.

0 Karma
Get Updates on the Splunk Community!

Leveraging Detections from the Splunk Threat Research Team & Cisco Talos

  Now On Demand  Stay ahead of today’s evolving threats with the combined power of the Splunk Threat Research ...

New in Splunk Observability Cloud: Automated Archiving for Unused Metrics

Automated Archival is a new capability within Metrics Management; which is a robust usage & cost optimization ...

Calling All Security Pros: Ready to Race Through Boston?

Hey Splunkers, .conf25 is heading to Boston and we’re kicking things off with something bold, competitive, and ...