Splunk Search

How to increase the results for append/appendcols?

uhkc777
Explorer

Hi,
In my query, i'm using append command to add the sub search with main search. But I'm getting max. of 50,000 events from sub search. How can I increase this limit?.

Thanks,

Tags (1)
0 Karma
1 Solution

sideview
SplunkTrust
SplunkTrust

There are other answers you might look at, for instance -- https://answers.splunk.com/answers/30678/append-and-max-results-50000.html

However the best thing is probably for you to post the syntax of your search and we can see if we can rewrite it to use faster and better SPL without the limits of append/join.

View solution in original post

0 Karma

sideview
SplunkTrust
SplunkTrust

There are other answers you might look at, for instance -- https://answers.splunk.com/answers/30678/append-and-max-results-50000.html

However the best thing is probably for you to post the syntax of your search and we can see if we can rewrite it to use faster and better SPL without the limits of append/join.

0 Karma

uhkc777
Explorer
 index=Test1 sourcetype=src1 earliest=@d 
| eval OD=ltrim(OD,"0"),Line=rtrim(Line,"") 
| table OD Line 
| append 
    [| search index=Test2 sourcetype=src2 earliest=@d 
    | eval created_date=strftime(strptime(CREATEDATE,"%Y%m%d"),"%Y-%m-%d"),today=strftime(now(),"%Y-%m-%d") 
    | where created_date=today 
    | dedup OD Line 
    | table OD Line 
    | rename OD as X_OD,Line as X_Line] 
| eval ODNUM=coalesce(OD,X_OD),LineNUM=coalesce(Line,X_Line) 
| stats values(OD) as A,values(X_OD) as B by ODNUM,LineNUM 
| appendcols 
    [| stats count] 
| eval Missing=case((isnotnull(A) AND isnull(B)),ODNUM) 
| stats dc(Missing)

It's simply calculating the missing ODs there in index 1 but missing in index2 by comparing OD&Line pair.

0 Karma

sideview
SplunkTrust
SplunkTrust

Yes this can absoilutely be rewritten as a disjunction plus a fair bit of "conditional eval".
Here you go, although I might still have a typo in here to fix.

(index=Test1 sourcetype=src1) OR (index=Test2 sourcetype=src2)
| eval created_date=if(sourcetype="src2",strftime(strptime(CREATEDATE,"%Y%m%d"),"%Y-%m-%d"),null())
| eval today=if(sourcetype="src2",strftime(now(),"%Y-%m-%d"),null())
| where sourcetype="src1" OR created_date=today 
| fields OD Line
| eval X_OD=if(sourcetype="src2",OD,null())
| eval X_Line=if(sourcetype="src2",Line,null())
| eval ODNUM=coalesce(OD,X_OD),LineNUM=coalesce(Line,X_Line) 
| stats values(OD) as A,values(X_OD) as B by ODNUM,LineNUM 
| appendcols 
    [| stats count] 
| eval Missing=case((isnotnull(A) AND isnull(B)),ODNUM) 
| stats dc(Missing)

the OD!=X_OD and the corresponding coalesce() can almost certainly be whittled down and kinda conjured away but I haven't done that here. the appendcols[| stats count].... I'm kinda pretending that's not there ~~but I see what it's doing.~~ but I think it's just a vestigial thing you can delete. Basically what I focused on here is making your main append go away.

0 Karma

uhkc777
Explorer

perfect. It's working. Thanks,

0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...