How to generate a search that will use values from my first search and find matching values on another index in a second search?

[LAcioffi]

Hello everyone!

I made a search, which returns some values like IP and Time and whatnot. Then, using these values, i want to make another search on another index for events with fields matching these values, and obtain some more information from the new events.
What's the best way to go about it? I tried using join but i've had no luck at all. Not sure if it's because i'm using it wrong or if it's not appropriate for the situation. Including both indexes at the start of the search is not feasible given the absurd size of the second index.

Any help is appreciated!!

[DalJeanis · Accepted solution]

You've asked the question in such a generic way that any answer we give might run you off a cliff. Effectiveness of a query depends on the data. So, there are plenty of different ways, but here are two of them.

The first is a subsearch -

    your second search here  [your first search in brackets here | table mysearchfield1 mysearchfield2...] 
   | whatever else you want to do

So, let's say the first search results go into a table that would be returning values for IP and _time. The outside search is going to find any record where the value of IP matches the IP on a row and the value of _time matches the _time on the table row.

The second is the map command

your first search here | table mysearchfield1 mysearchfield2...  
| map maxsearches=0 search="search your second search here IP=$mysearchfield1$ _time=$mysearchfield2$" 
| whatever else you want to do

This is going to take each result in the table and do an individual search for that combination.

The performance characteristics are going to differ slightly.

If you give us more detail on what you are doing, then we can help more.

[DalJeanis]

Great!

If the answer solved your problem, please mark the answer as accepted, so the question will not show as open.

[DalJeanis]

Here are some pseudo-code examples. some of the renames are not needed, but i wanted to make sure that you could see where each field was coming from. (Thus, I renamed _time to time1 or time2, etc).

If you need non-key information from BOTH searches, then the subsearch won't work. You can only pass keys from the subsearch to the outer search with that method.

SUBSEARCH

index=index2 other-search-stuff-2 
     [ search index=index1 KeyData1="myvalue" other-search-stuff-1 
     | rename IP1 as IP2 | table _time IP2 ] 
| rename IP2 as IP, _time as time2
| fields IP2 time2 KeyData2 OtherData2

JOIN

index=index1 KeyData1="myvalue" other-search-stuff-1 | rename _time as time1 | table IP1 time1 OtherData1
| join IP1 time1 
    [index=index2 other-search-stuff-2 | table IP2, _time, KeyData2, OtherData2 
    | rename IP2 as IP1, _time as time1] 
| fields IP1 time1 KeyData1 OtherData1 KeyData2 OtherData2

MAP

index=index1 KeyData1="myvalue" other-search-stuff-1 | rename _time as time1 | table IP1 time1 OtherData1 
| map maxsearches=0 search="search index=index2 other-search-stuff-2 IP2=$IP1$ _time=$time1$ 
    | table KeyData2 OtherData2" 
| fields IP1 time1 KeyData1 OtherData1 KeyData2 OtherData2

[LAcioffi]

Hey!
Thanks for the super answer. Map seems to work, but i had to make some changes (just putting it here so future n00bs can see it too):

index=something search_stuff
|doing stuff
|table value_to_be_searched
|map search="search index=something_else wanna_know=$value_to_be_searched$|table wanna_know|doing stuff again"

I have one more question now though:
Say i had a value "cow_color", which i use in a map to search "cow_name". I make a table containing cow_name and cow_color, but cow_color is now blank, as if the value was 'lost' after the map. Is there any way to 'keep' it?

[LAcioffi]

Figured it out!
You just have to use
"|map search="blablabla" |eval cow_color = $cow_color$| table cow_color, cow_name"