Splunk Search

How to remove duplicates from results of two searches on two different fields that have some of the same values?

BaptVe
Path Finder

Hello !

I launch a search with append to put the results of two searches together on different fields, but then I would like to remove the duplicates on these results:

First LOG :

24/05/2016 11:33:19,719 (...) service id : one

one is the value of the field Service

Second LOG:

24/05/2016 11:38:33,688 (...) service id : two

two is the value of the field state

The two logs are written differently and these two service id have two different field names in Splunk.
I've appended the two results:

index=XXXX com="*xxxx*" service=*  | append [ search
index=XXXX com="*xxxx*"  state=* ] 
| where state!= service |stats list(state)

And I tried with where to show the list, but without success!

Any help is welcome 😄

Thanks !

0 Karma

ishaanshekhar
Communicator

You can modify your search like this...

index=XXXX com="xxxx" service= | rename service as state 
|append [ search index=XXXX com="xxxx" state= ] 
| dedup state |stats list(state)

Thanks
Ishaan

0 Karma

lukasz92
Communicator
  1. Make one field common. For example add to the second search: 'eval service=state'
  2. Use 'dedup service, index, com' (and all other fields that are the same, if there is a duplicate)
0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!