Splunk Search

How to correlate between three data sources?

Path Finder

I have three data sources that I need to correlate together, I'll simplify it for sake of example:

Index A:
_time, fieldA, fieldB, fieldC

Index B: (web logs):
src, uri_path

Index C:
src, usr

FieldA from Index A should show up within the uri_path of Index B (within similar time ranges), when it does, I need to correlate the src IP address of Index B with Index B, and pull back the usr from Index C. The end result needs to contain all fields from Index A, plus the "src" field from Index B, plus the "usr" field from Index C.

This seems like it would be fairly easy if it were possible to pass data from outer searches to subsearches, but that's not possible. I tried starting with Index C and using a subsearch searching Index B, and a subsearch within that subsearch to search Index A, which returns FieldA to this first subsearch, which then returns "src" to the main outersearch which returns a list of users, however it seems difficult if not impossible to return all the needed fields back to the outer search.

I feel like I must be missing something and that it should be easier to correlate this data. Any ideas?

Thanks

0 Karma

Esteemed Legend

In fact, you can pass data from outer searches to later searches but it is very memory/search-intensive and you must be careful. The command is map. Another piece to your puzzle is that the match() command can take a field-name as the 2nd value and it will use the values in that field for the comparison.

0 Karma

Explorer

I'd probably link up index A and B through either a stats or subsearch. Then use a join to map src to user. Based on the size of the data in index C having a search create a continuously updated/ing lookup might make sense.

0 Karma

Path Finder

Thanks for the reply. Can you go into more detail how I would link index A and B through a subsearch? I would have to start with index B being the main search, correct? I'm not sure how to match part of a string in index B with a field value in index A, and also bring back the rest of the corresponding fields in the matching event in index A.

I'm also confused by how this would work with stats as well.

Thanks again

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!