Splunk Search

How to increase the results for append/appendcols?

uhkc777
Explorer

Hi,
In my query, i'm using append command to add the sub search with main search. But I'm getting max. of 50,000 events from sub search. How can I increase this limit?.

Thanks,

Tags (1)
0 Karma
1 Solution

sideview
SplunkTrust
SplunkTrust

There are other answers you might look at, for instance -- https://answers.splunk.com/answers/30678/append-and-max-results-50000.html

However the best thing is probably for you to post the syntax of your search and we can see if we can rewrite it to use faster and better SPL without the limits of append/join.

View solution in original post

0 Karma

sideview
SplunkTrust
SplunkTrust

There are other answers you might look at, for instance -- https://answers.splunk.com/answers/30678/append-and-max-results-50000.html

However the best thing is probably for you to post the syntax of your search and we can see if we can rewrite it to use faster and better SPL without the limits of append/join.

View solution in original post

0 Karma

uhkc777
Explorer
 index=Test1 sourcetype=src1 earliest=@d 
| eval OD=ltrim(OD,"0"),Line=rtrim(Line,"") 
| table OD Line 
| append 
    [| search index=Test2 sourcetype=src2 earliest=@d 
    | eval created_date=strftime(strptime(CREATEDATE,"%Y%m%d"),"%Y-%m-%d"),today=strftime(now(),"%Y-%m-%d") 
    | where created_date=today 
    | dedup OD Line 
    | table OD Line 
    | rename OD as X_OD,Line as X_Line] 
| eval ODNUM=coalesce(OD,X_OD),LineNUM=coalesce(Line,X_Line) 
| stats values(OD) as A,values(X_OD) as B by ODNUM,LineNUM 
| appendcols 
    [| stats count] 
| eval Missing=case((isnotnull(A) AND isnull(B)),ODNUM) 
| stats dc(Missing)

It's simply calculating the missing ODs there in index 1 but missing in index2 by comparing OD&Line pair.

0 Karma

sideview
SplunkTrust
SplunkTrust

Yes this can absoilutely be rewritten as a disjunction plus a fair bit of "conditional eval".
Here you go, although I might still have a typo in here to fix.

(index=Test1 sourcetype=src1) OR (index=Test2 sourcetype=src2)
| eval created_date=if(sourcetype="src2",strftime(strptime(CREATEDATE,"%Y%m%d"),"%Y-%m-%d"),null())
| eval today=if(sourcetype="src2",strftime(now(),"%Y-%m-%d"),null())
| where sourcetype="src1" OR created_date=today 
| fields OD Line
| eval X_OD=if(sourcetype="src2",OD,null())
| eval X_Line=if(sourcetype="src2",Line,null())
| eval ODNUM=coalesce(OD,X_OD),LineNUM=coalesce(Line,X_Line) 
| stats values(OD) as A,values(X_OD) as B by ODNUM,LineNUM 
| appendcols 
    [| stats count] 
| eval Missing=case((isnotnull(A) AND isnull(B)),ODNUM) 
| stats dc(Missing)

the OD!=X_OD and the corresponding coalesce() can almost certainly be whittled down and kinda conjured away but I haven't done that here. the appendcols[| stats count].... I'm kinda pretending that's not there ~~but I see what it's doing.~~ but I think it's just a vestigial thing you can delete. Basically what I focused on here is making your main append go away.

0 Karma

uhkc777
Explorer

perfect. It's working. Thanks,

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!