Splunk Search

How do I pass an event's field value from a subsearch to an eval statement to retrieve another field?

junxianli
Explorer

How do I pass an event's field value into a subsearch to retrieve another field?

At the moment, I can't use join because the records at the other sourcetype racks up to millions. Due to limitation, the join command will only return a maximum of 50,000 results to perform the join.

I need a direct search, to eval an extra field for each event using a field from the event.

Example of something I am trying,
Placing employeeID, a field from every event of the main search, into subsearch

index=a sourcetype=sta | eval employeeAddress= [index=b sourcetype=stb empID=$employeeID$ | return empAddress]
Tags (2)
1 Solution

sideview
SplunkTrust
SplunkTrust

Well if you're trying to get field values out of Search A index=a sourcetype=sta, and you want to use the field values in there to run another search B, and A might run into the millions of rows, then you can't use a subsearch.

I do however think you have your subsearch syntax backwards. The "first" search Splunk runs is always the inner one, and if I'm reading your question right that would be the index=a sourcetype=sta search. Therefore I think your hypothetical subsearch would look like:

index=b sourcetype=stb [ search index=a sourcetype=sta | rename employeeID as empID | table empID] table empID empAddress]

But again, the inner search will get truncated at 50,000 rows as you say so you can't use subsearches, join, append, etc...

However here's the good news:

1) Just get all the events and let stats sort them out.

(index=b sourcetype=stb empAddress=* empID=* ) OR (index=a sourcetype=sta employeeID=*) | eval empID=if(isnotnull(employeeID),employeeId,empID) | stats values(empAddress) by empID

2) If the search above seems to slow (because it gets many events off disk), then just run it once, or maybe once a day/week, to put the employeeID to EmpAddress mapping in a file based lookup.

Then you'll be able to run very efficient lookups to go from ID to address for your searches going forward.

Basic idea is same as #1, except you tack something like | outputlookup employeeAddresses on the end.

further reading about lookups -

1) http://docs.splunk.com/Documentation/Splunk/6.2.2/Search/Useexternalfieldlookups

2) http://docs.splunk.com/Documentation/Splunk/6.2.2/Knowledge/Usefieldlookupstoaddinformationtoyoureve...

View solution in original post

sideview
SplunkTrust
SplunkTrust

Well if you're trying to get field values out of Search A index=a sourcetype=sta, and you want to use the field values in there to run another search B, and A might run into the millions of rows, then you can't use a subsearch.

I do however think you have your subsearch syntax backwards. The "first" search Splunk runs is always the inner one, and if I'm reading your question right that would be the index=a sourcetype=sta search. Therefore I think your hypothetical subsearch would look like:

index=b sourcetype=stb [ search index=a sourcetype=sta | rename employeeID as empID | table empID] table empID empAddress]

But again, the inner search will get truncated at 50,000 rows as you say so you can't use subsearches, join, append, etc...

However here's the good news:

1) Just get all the events and let stats sort them out.

(index=b sourcetype=stb empAddress=* empID=* ) OR (index=a sourcetype=sta employeeID=*) | eval empID=if(isnotnull(employeeID),employeeId,empID) | stats values(empAddress) by empID

2) If the search above seems to slow (because it gets many events off disk), then just run it once, or maybe once a day/week, to put the employeeID to EmpAddress mapping in a file based lookup.

Then you'll be able to run very efficient lookups to go from ID to address for your searches going forward.

Basic idea is same as #1, except you tack something like | outputlookup employeeAddresses on the end.

further reading about lookups -

1) http://docs.splunk.com/Documentation/Splunk/6.2.2/Search/Useexternalfieldlookups

2) http://docs.splunk.com/Documentation/Splunk/6.2.2/Knowledge/Usefieldlookupstoaddinformationtoyoureve...

stephanefotso
Motivator

I propose that you use the map command.

index=a sourcetype=sta|stats count by sourcetype |map search="search index=b sourcetype=stb empID=$employeeID$"|table empAddress

Take this as a template:

index=_internal sourcetype=* user=*|stats count by user sourcetype|map search="search index=_audit user=$user$"|table action
SGF

stefan1988
Path Finder

Perfect solution! But how could i use the $user$ token in a dashboard? A dashboard thinks it's an input while it is not.

0 Karma

nick405060
Motivator
$$user$$
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...