We have indexed access logs into index="mpsapp", When we do a stats search or filter any records for these data for a particular month, it's extremely slow (took more than 1.5 hours for first query). Can you please tell us how to optimize this query?
Sample Splunk Queries :
Total Events for the month of October is : 355,925,951 events (10/1/14 12:00:00.000 AM to 11/1/14 12:00:00.000 AM)
Splunk Version : 6.1.4
Is there any configuration level optimization required to speed-up the query response? Please share your suggestions.
Hi dhavamanis,
try to use index
and any of the metadata
http://docs.splunk.com/Documentation/Splunk/6.2.0/SearchReference/Metadata fields ( host, source or sourcetype) in your searches. Try to filter the fields as tight as possible in your base search or use fields
http://docs.splunk.com/Documentation/Splunk/6.2.0/SearchReference/Fields#Examples as @aljohnson_splunk mentioned.
Do not use NOT
searches, rather search for events that you want and need.
Also, take a look at this great Answer of @jrodman about How do optimizations for field-based searches work http://answers.splunk.com/answers/172275/how-do-optimizations-for-field-based-searches-work.html#ans...
hope this helps to understand the possibilities to speed up your searches...
cheers, MuS
You can improve the performance by 10 X times by using Splunk meta data fields. I can help you in that please contact me in fiverr or Email (hurdlej1@gmail.com)
https://www.fiverr.com/s2/affc9b7a8a
https://www.fiverr.com/s2/608e8ed73f?utm_source=CopyLink_Mobile
Hi dhavamanis,
try to use index
and any of the metadata
http://docs.splunk.com/Documentation/Splunk/6.2.0/SearchReference/Metadata fields ( host, source or sourcetype) in your searches. Try to filter the fields as tight as possible in your base search or use fields
http://docs.splunk.com/Documentation/Splunk/6.2.0/SearchReference/Fields#Examples as @aljohnson_splunk mentioned.
Do not use NOT
searches, rather search for events that you want and need.
Also, take a look at this great Answer of @jrodman about How do optimizations for field-based searches work http://answers.splunk.com/answers/172275/how-do-optimizations-for-field-based-searches-work.html#ans...
hope this helps to understand the possibilities to speed up your searches...
cheers, MuS
Thanks!. We have accelerated the report and added to Dashboard. Using the summary index and its fetching results very fast.
We have done the following things after doing R & D.
1.Changed date range from real time to today.
2.Set dashboard refresh time to every 5 minutes.
3.Summary indexing
4.Report acceleration
5.Scheduled this search every 5 minutes so it will save in the cache.
6.Search query optimization.
7.Auto restart splunk daily at 2:00 AM UTC so that memory will be released.
8.Set high priority to this dashboard.
7.Set high priority to this scheduled search.
8.Run stats tables first then start charts.
9.Changed the delimer of raw data from text files method to new way which will reduce the time while converting raw data to fields of delimiting proccess.
10.Reduce the number of indexes and source type
After all this my dashboards loading time reduced from 3 minutes to less than 10 seconds.
Super fast
Do you ever use the fields
command to only retrieve the relevant fields ?
we haven't tried any fields command. Can you please give me some sample for fields command.
sourcetype=access_combined
Job Inspector: This search has completed and has returned 1,000 results by scanning 177,830 events in 14.393 seconds.
sourcetype=access_combined | fields clientip bytes action
Job Inpector: This search has completed and has returned 1,000 results by scanning 177,835 events in 3.508 seconds.
It's similar to using fast mode. Are you using fast mode?
Set search mode to adjust your search experience
Performance recommendations
How search types affect performance