Splunk Search

Lookup using an indexed log file

faustf
Communicator

Hi guys

I'm not an expert of Splunk.
I was wondering if I can use a lookup to reference fields that are stored into another log file (not csv) indexed in Splunk

Let me explain:
I have a log file indexed in Splunk:

col1,col2,GCD1,col4,col5
col1,col2,GCD2,col4,col5
col1,col2,GCD3,col4,col5

I've another file always indexed in Splunk:

graph [
    directed 1
    node [
        id 1
        Node "Node1"
    ]
    node [
        id 2
        Node "Node2"
    ]
    node [
        id 3
        Node "Node3"
    ]
]

I need a new field when I search for the first file that match the GDCID with the id in the second file

 col1,col2,GCD1,col4,col5,Node1
 col1,col2,GCD2,col4,col5,Node2
 col1,col2,GCD3,col4,col5,Node3

Is this possible?
Thanks

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

The general approach is something like this:

(search identifying data set 1) OR (search identifying data set 2) | stats values(field1) as field1 ... by common_field

The general assumption here is to have one event per "GCDn" on the left and one event per "id n" on the right, the stats stitches them together.

In your case you may need to do a bit of preprocessing, for example your first data set seems to have values like "GCD1" while the second data set appears to have values like "id 1" - those field values need to be harmonized before the stats, e.g. like this:

() OR () | eval common_field = case(expression identifying data set 1, replace(field_from_data_set_1, "GCD", ""), expression identifying data set 2, replace(field_from_data_set_1, "id ", ""), true(), "unknown id") | stats ...

More background: https://answers.splunk.com/answers/129424/how-to-compare-fields-over-multiple-sourcetypes-without-jo...
Even more background: https://wiki.splunk.com/Virtual_.conf March 2016 talk "Best practices around grouping and aggregating data from different search results"

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Persistent Queue at TcpOut — One of Splunk's Most Practical Features

Splunk introduced persistent queueing at the tcpout layer as one of the most practical resilience features in ...

Skip the Awkward Silence: Have a .conf-ersation at .conf26

Picture this. You arrive at .conf26 already having your socializing and networking plans mapped out. No ...

Rethinking Zero Trust: From Product Purchases to Logical Control Evidence

Implementing Zero Trust (ZT) across complex environments often falters at the very beginning due to a ...