topic Is there an alternative to subsearch or a way to raise the results limit? in Splunk Search

Is there an alternative to subsearch or a way to raise the results limit?

rlough — Wed, 28 Jan 2015 22:47:48 GMT

So I finally got my query to work only to find out that subsearch has a limit to 10,000 results! Is there a way to raise this limit or an accepted alternative method to getting around a limit? I'm dealing with hundreds of thousands of results.

Example of what my query looks like:

index=*ind* source=src1.log [search index=*ind* source=src2.log | table FIELD] | join usetime=true earlier=false FIELD [search index=*ind* source=src3.log] | table FIELD

I've tried to filter out results with more tags, but I can't get anywhere near 10,000 and this query works perfectly aside from that (minus the time it takes for it to parse...).

Any help would be greatly appreciated!!

Re: Is there an alternative to subsearch or a way to raise the results limit?

martin_mueller — Thu, 29 Jan 2015 00:05:57 GMT

Are you looking for FIELD values present in all three sources? Basically do this http://answers.splunk.com/answers/211727/how-to-edit-my-eventstats-search-to-keep-only-non.html or this http://answers.splunk.com/answers/211689/how-can-i-filter-out-unique-values-from-a-shared-f-1.html only with three sources?

Re: Is there an alternative to subsearch or a way to raise the results limit?

lguinn2 — Thu, 29 Jan 2015 01:44:41 GMT

Yes, there are often ways to get around using subsearches. I try to avoid subsearches both because of the limitations and because they are relatively slow. I also find that people with a database background (like me) tend to jump to subsearches when they really aren't necessary in Splunk. For example, your search can be done, I think, with this:

index=ind (source=src1.log OR source=src2.log or source=src3.log)
| eventstats count(eval(source=="src2.log")) as src2Count   count(eval(source=="src1.log")) as src1Count by FIELD
| where src2Count > 0 AND src1Count > 0 AND source!=src2.log
| eval src1_Time = if(source=="src1.log",_time,null())
| eval src3_Time = if(source=="src3.log",_time,null())
| stats latest(src1_time) as latest_src1  earliest(src3_time) as earliest_src3  
        count(isnotnull(src1_Time)) as src1Count   count(isnotnull(src3_Time)) as src3Count by FIELD
| where latest_src1 < earliest_src3 AND src1Count > 0 AND src3Count > 0
| table FIELD

However, I think there might be an even faster - and easier way - if you described what you are trying to do!

HTH

Re: Is there an alternative to subsearch or a way to raise the results limit?

rlough — Thu, 29 Jan 2015 19:28:45 GMT

Wow, that runs much faster! Thank you for pointing me in the right direction!

I hate that I have to use three sources, but it seems to be the only way. Thank you so much for the help though!

Re: Is there an alternative to subsearch or a way to raise the results limit?

MuS — Thu, 29 Jan 2015 20:39:15 GMT

in addition to @lguinn 's answer, if you want to learn more on this topic, check out this answer http://answers.splunk.com/answers/129424/how-to-compare-fields-over-multiple-sourcetypes-without-join-append-or-use-of-subsearches.html

cheers, MuS

Re: Is there an alternative to subsearch or a way to raise the results limit?

rlough — Thu, 29 Jan 2015 21:52:56 GMT

Wow! Thank you so much, this has completely changed how I'll query in the future. My query now works perfectly and super speedy! (Seriously, with subsearches it was taking 10 minutes to run, now it takes less than one.)
I'll now handle subsearches like I would handle a plague haha.