Splunk Search

How to dedup non-overlapping fields in separate sources?

yuanliu
SplunkTrust
SplunkTrust

I have two different sources with different fields.  Let's call them sourcetypeA and sourcetypeB.  Some fields that I wanted to dedup do not overlap.  Let's say sfieldA only exists in sourcetypeA, sfieldB only exists in sourcetypeB.  My intention is to have a single search (without append) to return events from both sources that contain unique sfieldA in sourcetypeA and unique sfieldB in sourcetypeB.

I was initially surprised that the following returned no event:

sourcetype = sourcetypeA OR sourcetype = sourcetype B
| dedup sfieldA sfieldB

Then, I realized that this is to ask for dedup on nonexistent keys.  My question is, then: Is there a syntax to express my intent?

Labels (2)
Tags (1)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

The method requires creating a common field.

sourcetype = sourcetypeA OR sourcetype = sourcetype B
| eval sfield = coalesce(sfieldA, sfieldB)
| dedup sfield

The coalesce function sets sfield to whichever field of sfieldA and sfieldB exists in the current event.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

The method requires creating a common field.

sourcetype = sourcetypeA OR sourcetype = sourcetype B
| eval sfield = coalesce(sfieldA, sfieldB)
| dedup sfield

The coalesce function sets sfield to whichever field of sfieldA and sfieldB exists in the current event.

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...