Splunk Search

How eliminate records from a search that appear twice only?

emipintus
Explorer

Hi, I have a search which returns a list of records, some of them have a duplicate Value. Here's an example of the output of my search:

TAG            VALUE
Running time(s) 8281    
Running time(s) 8281    
Running time(s) 8281    
Running time(s) 8906    
Running time(s) 8906    
Running time(s) 9672    
Running time(s) 10531
Running time(s) 10719
Running time(s) 13516

I need to keep all the values but those which appear twice. So, in the above example, I need to keep everything but 8906 because it appears twice. I tried with dedup but I don't understand how to make it work. Apparently dedup is not able to do something like this. Do you have any ideas how I can get rid of this?

Thank you,
Emilia

Tags (2)
0 Karma
1 Solution

nickhills
Ultra Champion

I'll try to refine this if you can provide your actual search, but making a few guesses try this:

Tag=*tagname* | sort +_time |eventstats count(VALUE) as ct by VALUE|where ct!=2|table Tag Value
If my comment helps, please give it a thumbs up!

View solution in original post

nickhills
Ultra Champion

I'll try to refine this if you can provide your actual search, but making a few guesses try this:

Tag=*tagname* | sort +_time |eventstats count(VALUE) as ct by VALUE|where ct!=2|table Tag Value
If my comment helps, please give it a thumbs up!

nickhills
Ultra Champion

My guess was pretty close 🙂
Give that a go.

If my comment helps, please give it a thumbs up!
0 Karma

emipintus
Explorer

Thank you nickhillscpl, I tried it and it works like crazy! This really solved my problem. Thank you so much for your quick and useful help 🙂

0 Karma

nickhills
Ultra Champion

You mean you need to drop values which appear EXACTLY twice.. so if they appear 1, 3 or more times you want to keep them?

If my comment helps, please give it a thumbs up!
0 Karma

emipintus
Explorer

Exactly. If a value appears only once or three times (or even more) I need to keep it 🙂

0 Karma

nickhills
Ultra Champion

Can you post your search which generates the result table?

If my comment helps, please give it a thumbs up!
0 Karma

emipintus
Explorer

Sure. It is a very simple search:

Tag=*tagname* | sort +_time | table Tag Value

and then I obtain that very small table. Thank you 🙂

0 Karma
Get Updates on the Splunk Community!

Now Available: Cisco Talos Threat Intelligence Integrations for Splunk Security Cloud ...

At .conf24, we shared that we were in the process of integrating Cisco Talos threat intelligence into Splunk ...

Preparing your Splunk Environment for OpenSSL3

The Splunk platform will transition to OpenSSL version 3 in a future release. Actions are required to prepare ...

Easily Improve Agent Saturation with the Splunk Add-on for OpenTelemetry Collector

Agent Saturation What and Whys In application performance monitoring, saturation is defined as the total load ...