Splunk Search

How to dedup non-overlapping fields in separate sources?

yuanliu
SplunkTrust
SplunkTrust

I have two different sources with different fields.  Let's call them sourcetypeA and sourcetypeB.  Some fields that I wanted to dedup do not overlap.  Let's say sfieldA only exists in sourcetypeA, sfieldB only exists in sourcetypeB.  My intention is to have a single search (without append) to return events from both sources that contain unique sfieldA in sourcetypeA and unique sfieldB in sourcetypeB.

I was initially surprised that the following returned no event:

sourcetype = sourcetypeA OR sourcetype = sourcetype B
| dedup sfieldA sfieldB

Then, I realized that this is to ask for dedup on nonexistent keys.  My question is, then: Is there a syntax to express my intent?

Labels (2)
Tags (1)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

The method requires creating a common field.

sourcetype = sourcetypeA OR sourcetype = sourcetype B
| eval sfield = coalesce(sfieldA, sfieldB)
| dedup sfield

The coalesce function sets sfield to whichever field of sfieldA and sfieldB exists in the current event.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

The method requires creating a common field.

sourcetype = sourcetypeA OR sourcetype = sourcetype B
| eval sfield = coalesce(sfieldA, sfieldB)
| dedup sfield

The coalesce function sets sfield to whichever field of sfieldA and sfieldB exists in the current event.

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...