topic Re: How to remove unique rows from a table? Is there a command opposite to dedup? in Splunk Search

How to remove unique rows from a table? Is there a command opposite to dedup?

MayankSplunk — Tue, 25 Nov 2014 16:51:31 GMT

From my search and transaction command I get the following table. To further process my results, I want to remove the row with ID V3 because it does not exist for type B. Is there a command opposite to dedup?

Type      ID    Score
A         V1     123
B         V1     786
A         V2     45
B         V2     34
A         V3     95

Re: How to remove unique rows from a table? Is there a command opposite to dedup?

somesoni2 — Tue, 25 Nov 2014 17:12:40 GMT

This can be achieved in two steps. (may need adjustment per your search)

your search producing above | eventstats count by ID | where count > 1

This will remove all the records where ID appears only once (unique).

Re: How to remove unique rows from a table? Is there a command opposite to dedup?

MayankSplunk — Tue, 25 Nov 2014 17:14:48 GMT

Ok found a way to do it

stats count dc(Type) as sourcetypes by ID | search sourcetypes > 1

change sourcetypes > 0 to 1 to remove unique row

Re: How to remove unique rows from a table? Is there a command opposite to dedup?

martin_mueller — Tue, 25 Nov 2014 18:23:15 GMT

Wouldn't this keep all the rows?

Re: How to remove unique rows from a table? Is there a command opposite to dedup?

MayankSplunk — Tue, 25 Nov 2014 18:27:43 GMT

I have updated the answer.

Re: How to remove unique rows from a table? Is there a command opposite to dedup?

MayankSplunk — Tue, 25 Nov 2014 18:29:30 GMT

Thanks @somesoni2 - that works as well.