Splunk Search

Weighting events based on the source they originate from

mjones414
Contributor

I have a summary index that pulls in normalized data from 2 different sources (entirely different applications that catalog and categorize the data differently).  In situations where I have events in the summary index from both sources, they are 99.99% of the time duplicates of eachother, however source 1 has better data fidelity than source 2.  Lets say if I weighted High Fidelity source with a 1 and Low fidelity source with a 2, I'm trying to find a way to filter with a by clause on another field which both events have  (like device, or ip_address).  something logically like:

|where source=coalesce("sourcename1","sourcename2") by field

but where doesnt take a by clause.

In the past I've done similar things by coalescing each field I want with a case statement, but in this case there are quite a few and I'm wondering if there's a more efficient way of doing it.

any ideas on the best way to accomplish this?

 

Labels (2)
0 Karma
1 Solution

ITWhisperer
SplunkTrust
SplunkTrust
| eval fidelity=if(source="source 1", 1, 2)
| eventstats min(fidelity) as best by device
| where fidelity == best

View solution in original post

mjones414
Contributor

I owe you a lot of beers!

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust
| eval fidelity=if(source="source 1", 1, 2)
| eventstats min(fidelity) as best by device
| where fidelity == best
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...