Archive

How to increase the results for append/appendcols?

Explorer

Hi,
In my query, i'm using append command to add the sub search with main search. But I'm getting max. of 50,000 events from sub search. How can I increase this limit?.

Thanks,

Tags (1)
0 Karma
1 Solution

SplunkTrust
SplunkTrust

There are other answers you might look at, for instance -- https://answers.splunk.com/answers/30678/append-and-max-results-50000.html

However the best thing is probably for you to post the syntax of your search and we can see if we can rewrite it to use faster and better SPL without the limits of append/join.

View solution in original post

0 Karma

SplunkTrust
SplunkTrust

There are other answers you might look at, for instance -- https://answers.splunk.com/answers/30678/append-and-max-results-50000.html

However the best thing is probably for you to post the syntax of your search and we can see if we can rewrite it to use faster and better SPL without the limits of append/join.

View solution in original post

0 Karma

Explorer
 index=Test1 sourcetype=src1 earliest=@d 
| eval OD=ltrim(OD,"0"),Line=rtrim(Line,"") 
| table OD Line 
| append 
    [| search index=Test2 sourcetype=src2 earliest=@d 
    | eval created_date=strftime(strptime(CREATEDATE,"%Y%m%d"),"%Y-%m-%d"),today=strftime(now(),"%Y-%m-%d") 
    | where created_date=today 
    | dedup OD Line 
    | table OD Line 
    | rename OD as X_OD,Line as X_Line] 
| eval ODNUM=coalesce(OD,X_OD),LineNUM=coalesce(Line,X_Line) 
| stats values(OD) as A,values(X_OD) as B by ODNUM,LineNUM 
| appendcols 
    [| stats count] 
| eval Missing=case((isnotnull(A) AND isnull(B)),ODNUM) 
| stats dc(Missing)

It's simply calculating the missing ODs there in index 1 but missing in index2 by comparing OD&Line pair.

0 Karma

SplunkTrust
SplunkTrust

Yes this can absoilutely be rewritten as a disjunction plus a fair bit of "conditional eval".
Here you go, although I might still have a typo in here to fix.

(index=Test1 sourcetype=src1) OR (index=Test2 sourcetype=src2)
| eval created_date=if(sourcetype="src2",strftime(strptime(CREATEDATE,"%Y%m%d"),"%Y-%m-%d"),null())
| eval today=if(sourcetype="src2",strftime(now(),"%Y-%m-%d"),null())
| where sourcetype="src1" OR created_date=today 
| fields OD Line
| eval X_OD=if(sourcetype="src2",OD,null())
| eval X_Line=if(sourcetype="src2",Line,null())
| eval ODNUM=coalesce(OD,X_OD),LineNUM=coalesce(Line,X_Line) 
| stats values(OD) as A,values(X_OD) as B by ODNUM,LineNUM 
| appendcols 
    [| stats count] 
| eval Missing=case((isnotnull(A) AND isnull(B)),ODNUM) 
| stats dc(Missing)

the OD!=X_OD and the corresponding coalesce() can almost certainly be whittled down and kinda conjured away but I haven't done that here. the appendcols[| stats count].... I'm kinda pretending that's not there ~~but I see what it's doing.~~ but I think it's just a vestigial thing you can delete. Basically what I focused on here is making your main append go away.

0 Karma

Explorer

perfect. It's working. Thanks,

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!