Splunk Search

How do I pass an event's field value from a subsearch to an eval statement to retrieve another field?

junxianli
Explorer

How do I pass an event's field value into a subsearch to retrieve another field?

At the moment, I can't use join because the records at the other sourcetype racks up to millions. Due to limitation, the join command will only return a maximum of 50,000 results to perform the join.

I need a direct search, to eval an extra field for each event using a field from the event.

Example of something I am trying,
Placing employeeID, a field from every event of the main search, into subsearch

index=a sourcetype=sta | eval employeeAddress= [index=b sourcetype=stb empID=$employeeID$ | return empAddress]
Tags (2)
1 Solution

sideview
SplunkTrust
SplunkTrust

Well if you're trying to get field values out of Search A index=a sourcetype=sta, and you want to use the field values in there to run another search B, and A might run into the millions of rows, then you can't use a subsearch.

I do however think you have your subsearch syntax backwards. The "first" search Splunk runs is always the inner one, and if I'm reading your question right that would be the index=a sourcetype=sta search. Therefore I think your hypothetical subsearch would look like:

index=b sourcetype=stb [ search index=a sourcetype=sta | rename employeeID as empID | table empID] table empID empAddress]

But again, the inner search will get truncated at 50,000 rows as you say so you can't use subsearches, join, append, etc...

However here's the good news:

1) Just get all the events and let stats sort them out.

(index=b sourcetype=stb empAddress=* empID=* ) OR (index=a sourcetype=sta employeeID=*) | eval empID=if(isnotnull(employeeID),employeeId,empID) | stats values(empAddress) by empID

2) If the search above seems to slow (because it gets many events off disk), then just run it once, or maybe once a day/week, to put the employeeID to EmpAddress mapping in a file based lookup.

Then you'll be able to run very efficient lookups to go from ID to address for your searches going forward.

Basic idea is same as #1, except you tack something like | outputlookup employeeAddresses on the end.

further reading about lookups -

1) http://docs.splunk.com/Documentation/Splunk/6.2.2/Search/Useexternalfieldlookups

2) http://docs.splunk.com/Documentation/Splunk/6.2.2/Knowledge/Usefieldlookupstoaddinformationtoyoureve...

View solution in original post

sideview
SplunkTrust
SplunkTrust

Well if you're trying to get field values out of Search A index=a sourcetype=sta, and you want to use the field values in there to run another search B, and A might run into the millions of rows, then you can't use a subsearch.

I do however think you have your subsearch syntax backwards. The "first" search Splunk runs is always the inner one, and if I'm reading your question right that would be the index=a sourcetype=sta search. Therefore I think your hypothetical subsearch would look like:

index=b sourcetype=stb [ search index=a sourcetype=sta | rename employeeID as empID | table empID] table empID empAddress]

But again, the inner search will get truncated at 50,000 rows as you say so you can't use subsearches, join, append, etc...

However here's the good news:

1) Just get all the events and let stats sort them out.

(index=b sourcetype=stb empAddress=* empID=* ) OR (index=a sourcetype=sta employeeID=*) | eval empID=if(isnotnull(employeeID),employeeId,empID) | stats values(empAddress) by empID

2) If the search above seems to slow (because it gets many events off disk), then just run it once, or maybe once a day/week, to put the employeeID to EmpAddress mapping in a file based lookup.

Then you'll be able to run very efficient lookups to go from ID to address for your searches going forward.

Basic idea is same as #1, except you tack something like | outputlookup employeeAddresses on the end.

further reading about lookups -

1) http://docs.splunk.com/Documentation/Splunk/6.2.2/Search/Useexternalfieldlookups

2) http://docs.splunk.com/Documentation/Splunk/6.2.2/Knowledge/Usefieldlookupstoaddinformationtoyoureve...

stephanefotso
Motivator

I propose that you use the map command.

index=a sourcetype=sta|stats count by sourcetype |map search="search index=b sourcetype=stb empID=$employeeID$"|table empAddress

Take this as a template:

index=_internal sourcetype=* user=*|stats count by user sourcetype|map search="search index=_audit user=$user$"|table action
SGF

stefan1988
Path Finder

Perfect solution! But how could i use the $user$ token in a dashboard? A dashboard thinks it's an input while it is not.

0 Karma

nick405060
Motivator
$$user$$
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...