How to search unique values in field1 after removing duplicates and common values found in field2?

[raju4244]

Dear All,

I have one question. I have the data like below:

field1:

itema
itemb
itemb
itemc
itemd
iteme
iteme

field2:

itemc
itemd
itemd
iteme

I want to get the following in my search results:

Unique Items which are present in field1 (after removing duplicates and after removing common items present in both fields)

[somesoni2 · Accepted solution]

Try something like this

index=foo sourcetype=sourcetype1 OR sourcetype=sourcetype2 | eval commonfield=coalesce(field1,field2) | stats values(sourcetype) as sourcetypes by commonfield | where mvindex(sourcetypes)=1 AND sourcetypes="sourcetype1" | table commonfield

[raju4244]

this worked after some modification, thanks to you.

[ppablo]

Hi @raju4244

As @martin_mueller mentioned, it'd be helpful if you could share the final search you used that worked for you. This forum isn't just for individuals, it's for the greater Splunk community at large 🙂

[martin_mueller]

Do post what you modified for others to use.

[richgalloway]

Try this:

index=foo sourcetype=baz | dedup field2 | eval fieldc=field2 | join fieldc [ search index=foo sourcetype=bar | dedup field1 | eval fieldc=field1] | table field1

[raju4244]

it is only listing common items, not an unique items present in field1

[richgalloway]

Then I gave you the wrong join type. Try ... | join type=left fieldc [ search index=foo sourcetype=bar | dedup field1 |....

[richgalloway]

Are both fields present in the same event or are they in different events?

[raju4244]

They are from different sourcetype, i m trying to club them into one list.

Thanks.