Splunk Search

How to correlate between three data sources?

ehowardl3
Path Finder

I have three data sources that I need to correlate together, I'll simplify it for sake of example:

Index A:
_time, fieldA, fieldB, fieldC

Index B: (web logs):
src, uri_path

Index C:
src, usr

FieldA from Index A should show up within the uri_path of Index B (within similar time ranges), when it does, I need to correlate the src IP address of Index B with Index B, and pull back the usr from Index C. The end result needs to contain all fields from Index A, plus the "src" field from Index B, plus the "usr" field from Index C.

This seems like it would be fairly easy if it were possible to pass data from outer searches to subsearches, but that's not possible. I tried starting with Index C and using a subsearch searching Index B, and a subsearch within that subsearch to search Index A, which returns FieldA to this first subsearch, which then returns "src" to the main outersearch which returns a list of users, however it seems difficult if not impossible to return all the needed fields back to the outer search.

I feel like I must be missing something and that it should be easier to correlate this data. Any ideas?

Thanks

0 Karma

woodcock
Esteemed Legend

In fact, you can pass data from outer searches to later searches but it is very memory/search-intensive and you must be careful. The command is map. Another piece to your puzzle is that the match() command can take a field-name as the 2nd value and it will use the values in that field for the comparison.

0 Karma

mrunals
Path Finder

I'd probably link up index A and B through either a stats or subsearch. Then use a join to map src to user. Based on the size of the data in index C having a search create a continuously updated/ing lookup might make sense.

0 Karma

ehowardl3
Path Finder

Thanks for the reply. Can you go into more detail how I would link index A and B through a subsearch? I would have to start with index B being the main search, correct? I'm not sure how to match part of a string in index B with a field value in index A, and also bring back the rest of the corresponding fields in the matching event in index A.

I'm also confused by how this would work with stats as well.

Thanks again

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...