How to increase the results for append/appendcols?

[uhkc777]

Hi,
In my query, i'm using append command to add the sub search with main search. But I'm getting max. of 50,000 events from sub search. How can I increase this limit?.

Thanks,

[sideview · Accepted solution]

There are other answers you might look at, for instance -- https://answers.splunk.com/answers/30678/append-and-max-results-50000.html

However the best thing is probably for you to post the syntax of your search and we can see if we can rewrite it to use faster and better SPL without the limits of append/join.

[uhkc777]

 index=Test1 sourcetype=src1 earliest=@d 
| eval OD=ltrim(OD,"0"),Line=rtrim(Line,"") 
| table OD Line 
| append 
    [| search index=Test2 sourcetype=src2 earliest=@d 
    | eval created_date=strftime(strptime(CREATEDATE,"%Y%m%d"),"%Y-%m-%d"),today=strftime(now(),"%Y-%m-%d") 
    | where created_date=today 
    | dedup OD Line 
    | table OD Line 
    | rename OD as X_OD,Line as X_Line] 
| eval ODNUM=coalesce(OD,X_OD),LineNUM=coalesce(Line,X_Line) 
| stats values(OD) as A,values(X_OD) as B by ODNUM,LineNUM 
| appendcols 
    [| stats count] 
| eval Missing=case((isnotnull(A) AND isnull(B)),ODNUM) 
| stats dc(Missing)

It's simply calculating the missing ODs there in index 1 but missing in index2 by comparing OD&Line pair.

[sideview]

Yes this can absoilutely be rewritten as a disjunction plus a fair bit of "conditional eval".
Here you go, although I might still have a typo in here to fix.

(index=Test1 sourcetype=src1) OR (index=Test2 sourcetype=src2)
| eval created_date=if(sourcetype="src2",strftime(strptime(CREATEDATE,"%Y%m%d"),"%Y-%m-%d"),null())
| eval today=if(sourcetype="src2",strftime(now(),"%Y-%m-%d"),null())
| where sourcetype="src1" OR created_date=today 
| fields OD Line
| eval X_OD=if(sourcetype="src2",OD,null())
| eval X_Line=if(sourcetype="src2",Line,null())
| eval ODNUM=coalesce(OD,X_OD),LineNUM=coalesce(Line,X_Line) 
| stats values(OD) as A,values(X_OD) as B by ODNUM,LineNUM 
| appendcols 
    [| stats count] 
| eval Missing=case((isnotnull(A) AND isnull(B)),ODNUM) 
| stats dc(Missing)

the OD!=X_OD and the corresponding coalesce() can almost certainly be whittled down and kinda conjured away but I haven't done that here. the appendcols[| stats count].... I'm kinda pretending that's not there ~~but I see what it's doing.~~ but I think it's just a vestigial thing you can delete. Basically what I focused on here is making your main append go away.

[uhkc777]

perfect. It's working. Thanks,