I am doing a simple search:
index=pqr host=xyz* NOT TYPE="*ABCDE*" | fields X, Y | timechart limit=0 span=10m count, avg(X) by Y
on a two week period.
The search has been running for more than two hours now and is still only at about 60%. Is there a way to make it run faster?
Also sometimes I get
'search has been remotely
cancelled or expired'
What does that mean? how do I prevent a search from getting cancelled?
Also, do not use
* searches. Since Splunk will have to read in your example, all events containing
host fields from disk and compare if the values starts with
Take a good lock at the slide here http://conf.splunk.com/session/2015/conf2015_JHarty_DuncanTurnbull_Splunk_UsingSplunkSearchLanguage_... to learn more about Search Efficiency Optimisation.
Give this a try
index=pqr host=xyz* TYPE!="*ABCDE*" | table _time, X, Y | timechart limit=0 span=10m count, avg(X) by Y
Also, if the data span is to a limited sourcetypes, please add them as well as filter.
when i tried above, its running but all fields are staying blank. So i replaced 'table _time' with 'fields' and it's working better.
Is there anyway I can measure/compare the performance of the two search queries in terms of total approx. run time?
YOu can check the job inspector (there is a job menu just below the search bar towards the right side) which shows the amount of time taken by the search.
This looks like a good case for a summary index. Populate the index every 10 minutes with your stats, and then you search the summaries when you want to build a trend.
To answer your question directly, you can prevent it from happening by EITHER speeding up your search OR clicking on the
Jobs menu click
Send Job to Background and your job will NOT timeout and you can opt to receive an email when the job completes.