subsearch limit

[akawacz]

Hi,

How can i overcome subsearch limitation. I do not want to change limit in conf files. I have read that this can cause instability.

In my below search I want to find differences between two data sets. My subsearch is returing just 50.000 rows that is why search not working properly

index=A  earliest=-45d@| dedup IM_ID
|eval TYPE_OF_REPORT="Y"
|eval HASH=md5(ifnull(ASSIGNEE,"Empty").ifnull(ID,"Empty")
| table ID ASSIGNEE HASH  TYPE_OF_REPORT
|append [search index = B|eval TYPE_OF_REPORT="X"| table  ID ASSIGNEE HASH  TYPE_OF_REPORT] |stats values(*) AS * dc(TYPE_OF_REPORT) as t by HASH |where t=1 and match(TYPE_OF_REPORT,"Y").

Thanks

[sundareshr · Accepted solution]

How about something like this (not tested)

(index-A OR index=B) | dedup IM_ID index
| eval hash=md5(ifnull(ASSIGNEE,"Empty").ifnull(ID,"Empty")
| streamstats window=1 list(eval(if(index="A", "Y", "X"))) as TYPE_OF_REPORT by index
| table ID ASSIGNEE HASH TYPE_OF_REPORT
| stats values(*) as * dc(TYPE_OF_REPORT) as t by hash
| where t=1 and match(TYPE_OF_REPORT, "Y")

[akawacz]

Hi

You gave me idea with combaning indexs by OR.

I have used eval with If instead of streamstats.

eval TYPE_OF_REPORT=if(index=="A","X","Y")

Thank you