Increase subsearch results limits when using join

[manish_singh_77]

Hi All,

I have data coming in from different indexes and am joining them on the common field. Data is huge, so when I join the sub searches it doesn't give the full results. Can we increase the subsearches results count to max when using join command.

Regard
Manish Singh

[woodcock]

The best answers is: Don't. Use stats instead. Show some sample events in each data set, then show us a mockup of the desired results and describe any nuances. We will get you a non- join method. See here for general approaches:

https://conf.splunk.com/files/2016/slides/let-stats-sort-them-out-building-complex-result-sets-that-...

[manish_singh_77]

Hi Niket/Woodcock,

I am trying to bring in data from different indexes and then trying to match with the application inventory so that I will have one table where in I could see the check list of 4-5 columns which am looking for, like whether we are getting any data in splunk for app, infra and alerts from alerting tool in splunk.

Here is the output would be like

Application Name  Infra_Data  App_Data   Events_Data   Appdynamics_Data
ABC               Yes         No         Yes           Yes

Here is the mock query,

| inputlookup Application_Inventory
| where Application="PROD" 
| rename Server as host 
| join type=left host 
[search (index=linux_windows_os*) OR (index=xyz_*) OR (index=applications_data-*) OR (index=applications_info*)  earliest=-24h@h latest=now() 
| stats latest(_time) as latest_time values(host) as host by index 
| eval current_time=now() 
| eval Time_difference=(current_time-latest_time) 
| eval Validation=if(TimeDiff>86400,"No","Yes") 
| eval latest_time=strftime(latest_time,"%F %T"), current_time=strftime(current_time,"%F %T"), TimeDiff=strftime(TimeDiff,"%S") 
| dedup index 
| table index Validation host 
| eval Indexes=case(index like "%applicatons_%", "App_data",index like "%linux_windows_%","Infra Data"|mvexpand host|xyseries host Indexes Validation]|table App_data "Infra Data", Application_Component_Name, PlatformName|join type=left Application_Component[search index=*alerts_data* earliest=-24h@h latest=now()|eval AlertsData=if(Application_Component!= " ", "Yes","No")|table AlertsData, Application_Component|dedup Application_Component]|join type=left  Application_Component[|inputlookup AppDynamics_Data]

I have taken common fields to match the data with the Application_Inventory lookup.

Let me know if you guys have better options to achieve this or the question needs more clarification.

[manish_singh_77]

@ Niket & Woodcockl,

Do you have any workaround for this?

Regards
Manish Singh

[manish_singh_77]

@niketnilay @woodcock , hello mates, do you have any workaround for this?

[woodcock]

The code that you posted is broken. It has an unterminated case statement and mis-matched square brackets. It is hopelessly broken so we cannot help until you fix it.

[manish_singh_77]

Sorry, I should have posted this under comments section, my apologies..

[jeffland]

I moved it for you.

[jeffland]

You may want to have a look at this thread discussing alternatives to join and subsearches. If you must, subsearch limits are configured in limits.conf.

[manish_singh_77]

I have gone through that thread before and it is not helping me..

[niketn]

@manish_singh_777 if you have data in two separate indexes and you are using join it is highly possible you can bring in data from both the search at one place and use stats. However, please provide your existing search along with sample data and required output for us to assist you better. Please mock/anonymize any sensitive information before posting.

[jeffland]

Then check the appropriate sections in limits.conf and increase the subsearch result count. It should be the setting subsearch_maxout under the join stanza.