How can I optimize the performance of my search?

[m_vivek]

I am doing a simple search:

index=pqr host=xyz* NOT TYPE="*ABCDE*" | fields X, Y | timechart limit=0 span=10m count, avg(X) by Y

on a two week period.
The search has been running for more than two hours now and is still only at about 60%. Is there a way to make it run faster?

Also sometimes I get

'search has been remotely
cancelled or expired'

What does that mean? how do I prevent a search from getting cancelled?

[woodcock]

To answer your question directly, you can prevent it from happening by EITHER speeding up your search OR clicking on the Jobs menu click Send Job to Background and your job will NOT timeout and you can opt to receive an email when the job completes.

[muebel]

This looks like a good case for a summary index. Populate the index every 10 minutes with your stats, and then you search the summaries when you want to build a trend.

[m_vivek]

hi muebel, could you elaborate some. i'm pretty new to splunk .

[somesoni2]

Some reading about summary indexing available here.
http://docs.splunk.com/Documentation/Splunk/6.2.0/Knowledge/Usesummaryindexing

[somesoni2]

Give this a try

index=pqr host=xyz* TYPE!="*ABCDE*" | table _time, X, Y | timechart limit=0 span=10m count, avg(X) by Y

Also, if the data span is to a limited sourcetypes, please add them as well as filter.

[m_vivek]

when i tried above, its running but all fields are staying blank. So i replaced 'table _time' with 'fields' and it's working better.
Is there anyway I can measure/compare the performance of the two search queries in terms of total approx. run time?

[somesoni2]

YOu can check the job inspector (there is a job menu just below the search bar towards the right side) which shows the amount of time taken by the search.

[walkerhound]

According to this link: http://docs.splunk.com/Documentation/Splunk/6.3.0/Search/Writebettersearches
you should avoid using NOT wherever possible

[MuS]

Also, do not use * searches. Since Splunk will have to read in your example, all events containing host fields from disk and compare if the values starts with xyz.

Take a good lock at the slide here http://conf.splunk.com/session/2015/conf2015_JHarty_DuncanTurnbull_Splunk_UsingSplunkSearchLanguage_... to learn more about Search Efficiency Optimisation.

cheers, MuS