Hi All,
I have data coming in from different indexes and am joining them on the common field. Data is huge, so when I join the sub searches it doesn't give the full results. Can we increase the subsearches results count to max when using join command.
Regard
Manish Singh
The best answers is: Don't. Use stats
instead. Show some sample events in each data set, then show us a mockup of the desired results and describe any nuances. We will get you a non- join
method. See here for general approaches:
Hi Niket/Woodcock,
I am trying to bring in data from different indexes and then trying to match with the application inventory so that I will have one table where in I could see the check list of 4-5 columns which am looking for, like whether we are getting any data in splunk for app, infra and alerts from alerting tool in splunk.
Here is the output would be like
Application Name Infra_Data App_Data Events_Data Appdynamics_Data
ABC Yes No Yes Yes
Here is the mock query,
| inputlookup Application_Inventory
| where Application="PROD"
| rename Server as host
| join type=left host
[search (index=linux_windows_os*) OR (index=xyz_*) OR (index=applications_data-*) OR (index=applications_info*) earliest=-24h@h latest=now()
| stats latest(_time) as latest_time values(host) as host by index
| eval current_time=now()
| eval Time_difference=(current_time-latest_time)
| eval Validation=if(TimeDiff>86400,"No","Yes")
| eval latest_time=strftime(latest_time,"%F %T"), current_time=strftime(current_time,"%F %T"), TimeDiff=strftime(TimeDiff,"%S")
| dedup index
| table index Validation host
| eval Indexes=case(index like "%applicatons_%", "App_data",index like "%linux_windows_%","Infra Data"|mvexpand host|xyseries host Indexes Validation]|table App_data "Infra Data", Application_Component_Name, PlatformName|join type=left Application_Component[search index=*alerts_data* earliest=-24h@h latest=now()|eval AlertsData=if(Application_Component!= " ", "Yes","No")|table AlertsData, Application_Component|dedup Application_Component]|join type=left Application_Component[|inputlookup AppDynamics_Data]
I have taken common fields to match the data with the Application_Inventory
lookup.
Let me know if you guys have better options to achieve this or the question needs more clarification.
@ Niket & Woodcockl,
Do you have any workaround for this?
Regards
Manish Singh
@niketnilay @woodcock , hello mates, do you have any workaround for this?
The code that you posted is broken. It has an unterminated case
statement and mis-matched square brackets. It is hopelessly broken so we cannot help until you fix it.
Sorry, I should have posted this under comments section, my apologies..
I moved it for you.
You may want to have a look at this thread discussing alternatives to join
and subsearches. If you must, subsearch limits are configured in limits.conf.
I have gone through that thread before and it is not helping me..
@manish_singh_777 if you have data in two separate indexes and you are using join it is highly possible you can bring in data from both the search at one place and use stats. However, please provide your existing search along with sample data and required output for us to assist you better. Please mock/anonymize any sensitive information before posting.
Then check the appropriate sections in limits.conf and increase the subsearch result count. It should be the setting subsearch_maxout
under the join
stanza.