How to parse outer search to get earliest and latest time range of subsearch?

[shay]

Hi,

I am trying to create a search the looks for specific signatures detected on the IPS and then returns all related firewall and proxy logs, grouped by each related set of events. I have written the following query. It is returning the correct results but taking hours to run and looks like the time fields are not being parsed to the outer search:

(index=ips OR index=firewall OR index=proxy)
     [search  index=ips signature_id  IN (25007, 25008,25009)
     | eval earliest=_time-300
     | eval latest = _time+60
     | fields earliest latest src_ip]
transaction src_ip

The search is run over 90days. The inner search completes after around 30sec and returns 6 results.
I am wanting to run the outer search for each of the six results (5min before the IPS event to 1min after).

While the results a get are correct, the search took 10hrs to run. If I manually enter the earliest and latest and src_ip  into the following each result only takes around 2min:

(index=ips OR index=firewall OR index=proxy) earliest=X latest=Y src_ip=Z
transaction src_ip

So I think the outer search is being run 6 times for either 'All Time' or 'Last 90days'. Can anyone assist me with getting the earliest and latest to parse so that it only runs the query on a 6min range for each result of the inner search?

Thanks very much.

[isoutamo]

Can you try to get only
“| stats earliest_time(_time) as earliest latest_time(_time) as latest values(src_ip) “
from the Inner loop instead of those six separate earliest and latest?

[thambisetty]

yes, inner search takes time range from time range you chosen in time range picker. 

for example , if you choose last 90 days, inner search earliest time would be 90days-300seconds and latest time would now()+60 seconds.

I read your question properly understand that you want to pass different times based on your inner search results. is that right ?

[thambisetty]

index=firewall OR index=proxy 
    [ search index=ips signature_id IN (25007, 25008,25009) 
        [| gentimes start=-1 
        | addinfo 
        | eval earliest=info_min_time-300,latest=info_max_time+60 
        | table earliest,latest] 
    | stats count by src_ip 
    | table src_ip
        ]

[shay]

Thanks for your reply @thambisetty .

Unfortunately that isn't working. It is still taking many hours to run and now it returns 1000s of results. Results that do not match the times of the subsearch. It appears to be pulling the all the logs relating to any of the IPs from the last 90days.