Splunk Search

How eliminate records from a search that appear twice only?

emipintus
Explorer

Hi, I have a search which returns a list of records, some of them have a duplicate Value. Here's an example of the output of my search:

TAG            VALUE
Running time(s) 8281    
Running time(s) 8281    
Running time(s) 8281    
Running time(s) 8906    
Running time(s) 8906    
Running time(s) 9672    
Running time(s) 10531
Running time(s) 10719
Running time(s) 13516

I need to keep all the values but those which appear twice. So, in the above example, I need to keep everything but 8906 because it appears twice. I tried with dedup but I don't understand how to make it work. Apparently dedup is not able to do something like this. Do you have any ideas how I can get rid of this?

Thank you,
Emilia

Tags (2)
0 Karma
1 Solution

nickhills
Ultra Champion

I'll try to refine this if you can provide your actual search, but making a few guesses try this:

Tag=*tagname* | sort +_time |eventstats count(VALUE) as ct by VALUE|where ct!=2|table Tag Value
If my comment helps, please give it a thumbs up!

View solution in original post

nickhills
Ultra Champion

I'll try to refine this if you can provide your actual search, but making a few guesses try this:

Tag=*tagname* | sort +_time |eventstats count(VALUE) as ct by VALUE|where ct!=2|table Tag Value
If my comment helps, please give it a thumbs up!

nickhills
Ultra Champion

My guess was pretty close 🙂
Give that a go.

If my comment helps, please give it a thumbs up!
0 Karma

emipintus
Explorer

Thank you nickhillscpl, I tried it and it works like crazy! This really solved my problem. Thank you so much for your quick and useful help 🙂

0 Karma

nickhills
Ultra Champion

You mean you need to drop values which appear EXACTLY twice.. so if they appear 1, 3 or more times you want to keep them?

If my comment helps, please give it a thumbs up!
0 Karma

emipintus
Explorer

Exactly. If a value appears only once or three times (or even more) I need to keep it 🙂

0 Karma

nickhills
Ultra Champion

Can you post your search which generates the result table?

If my comment helps, please give it a thumbs up!
0 Karma

emipintus
Explorer

Sure. It is a very simple search:

Tag=*tagname* | sort +_time | table Tag Value

and then I obtain that very small table. Thank you 🙂

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...