Splunk Search

How to remove repeated duplicate values?

alexspunkshell
Contributor

I am trying to remove duplicates in my result using the |dedup command. Even though I am seeing 2 entries in my result.

Kindly help me to remove 1 duplicate.

 

alexspunkshell_0-1633093094982.png

 

Labels (2)
0 Karma
1 Solution

ITWhisperer
SplunkTrust
SplunkTrust
| stats count by title
| fields - count

View solution in original post

Lavender
Loves-to-Learn Everything

I too having same kind of issue . I have tried with your solution , but still I can see duplicate values . please help @ITWhisperer 

 

Lavender_0-1681983560162.png

 

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

I have no idea what SPL you used to create that output since you didn't share it. Having said that, given that you apparently have three occurrences of the same string, perhaps your base data is at fault, or perhaps you have trailing spaces?

0 Karma

Lavender
Loves-to-Learn Everything

I got the solution . 

mvexpand doesn't work because the field is not a multi-value field.  It's a single-value field with embedded newlines.  Tried using the split function to break up the field then mvexpand  and it works

0 Karma

Azeemering
Builder

This field contains multiple duplicate values I guess.

You can remove it like this:

index=graphsecurityalert | mvexpand title | dedup title | table title

Check to see which events contain those multivalues:

index=graphsecurityalert | eval c=mvcount(title) | table c

 

ITWhisperer
SplunkTrust
SplunkTrust
| eval title=mvdedup(title)

ITWhisperer
SplunkTrust
SplunkTrust
| stats count by title
| fields - count
Get Updates on the Splunk Community!

Enhance Your Splunk App Development: New Tools & Support

UCC FrameworkAdd-on Builder has been around for quite some time. It helps build Splunk apps faster, but it ...

Prove Your Splunk Prowess at .conf25—No Prereqs Required!

Your Next Big Security Credential: No Prerequisites Needed We know you’ve got the skills, and now, earning the ...

Splunk Observability Cloud's AI Assistant in Action Series: Observability as Code

This is the sixth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...