Splunk Search

How to dedup non-overlapping fields in separate sources?

yuanliu
SplunkTrust
SplunkTrust

I have two different sources with different fields.  Let's call them sourcetypeA and sourcetypeB.  Some fields that I wanted to dedup do not overlap.  Let's say sfieldA only exists in sourcetypeA, sfieldB only exists in sourcetypeB.  My intention is to have a single search (without append) to return events from both sources that contain unique sfieldA in sourcetypeA and unique sfieldB in sourcetypeB.

I was initially surprised that the following returned no event:

sourcetype = sourcetypeA OR sourcetype = sourcetype B
| dedup sfieldA sfieldB

Then, I realized that this is to ask for dedup on nonexistent keys.  My question is, then: Is there a syntax to express my intent?

Labels (1)
Tags (1)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

The method requires creating a common field.

sourcetype = sourcetypeA OR sourcetype = sourcetype B
| eval sfield = coalesce(sfieldA, sfieldB)
| dedup sfield

The coalesce function sets sfield to whichever field of sfieldA and sfieldB exists in the current event.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

The method requires creating a common field.

sourcetype = sourcetypeA OR sourcetype = sourcetype B
| eval sfield = coalesce(sfieldA, sfieldB)
| dedup sfield

The coalesce function sets sfield to whichever field of sfieldA and sfieldB exists in the current event.

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...

Brains, Bytes, and Boston: Learn from the Best at .conf25

When you think of Boston, you might picture colonial charm, world-class universities, or even the crack of a ...