Hello,
I have upgraded Splunk Enterprise to 7.0.1. One of the search query is taking ages to finish it. Same query finished quickly in Splunk 6.x.
Splunk 6.6.1 = 5 secs
Splunk 7.0.1 = 26 mins (still running)
Does anyone have encounter such situation or have idea for this behaviour in Splunk 7.
In my case, I have added "| noop search_optimization=false" at the last line which makes my query fast. You can give a try.
@sarfarajsayyad @agoyal I am also getting same issue. Were you able to resolve the issue? If yes, what is the solution? Is this issue is specific to new version 7.1.0?
In my case I have added "| noop search_optimization=false" at the end of the line to resolve the issue. It worked for me, you can give a try.
@agoyal, you second search should be part of your base search
<yourBaseSearch> "*concurrent mode failure*"
| stats count as FAIL
Have you tried it this way?
Same behavior I see on my setup. My query was running fine on 6.5.2 (within 2 mins) and now it is taking more time (90 mins) on 7.0.2. Is there any change happened on 7.0.x? I don't see any error msg on splunkd log too. There is no clue.
@sarfarajsayyad , would it be possible for you to share the query?
That query is having some customer specific info. Cant share here.
My point is, the same query is running fine on 6.5.2 and not on 7.0.2. Believe me, I have installed 7.0.2 Splunk enterprise on a new machine without data. Still, it's taking more time. Looks like its nothing related to data. Something is changed in 7.0.x.
JFYI - In my query, I have 20+ joined on various indexes/lookup.
@sarfarajsayyad if you have valid Splunk Entitlement, reach out to Splunk Support.
If I use "|noop search_optimization=false" at the end of my query its giving me result very fast. After cross-checking I got the same result.
Is there any impact of "|noop search_optimization=false" ?
I see lot of below messages
02-22-2018 04:06:41.512 INFO SearchPipeline - Command='eval' doesnt have raw field
02-22-2018 04:06:41.512 INFO SearchPipeline - Command='inputlookup' doesnt have raw field
02-22-2018 04:06:41.512 INFO SearchPipeline - Command='inputlookup' doesnt have raw field
02-22-2018 04:06:41.512 INFO SearchPipeline - Command='eval' doesnt have raw field
02-22-2018 04:06:41.512 INFO SearchPipeline - Command='search' doesnt have raw field
02-22-2018 04:06:41.512 INFO SortOperator - maxmem = 209715200
In the first search its written before 17/1/2018 from how long do you have data in your system?
file is created on 17/01/2018 06:27:12. So its having data for few hours.
same file is taking by both Splunk version. I am having feeling that as time is specified as "All Time" , Splunk 7 is not specifically looking for specified file. It's trying to find out data from the begnning of Splunk Time (1st Jan 1970).
Search is still running.
That's correct. There are a small handful of fields that are extracted at index-time, and unless you've done some intentional work to change that on your system, then the field service_name
will not be one of them. So now Splunk is searching back through all indexed data looking for any possible matches. Your best bets to make this search run more efficiently are to narrow the time window and specify the index(es) in which you'd like to search. Here's a good guide to provide more of an overview:
http://docs.splunk.com/Documentation/SplunkCloud/6.6.3/Search/Writebettersearches#Tips_for_tuning_yo...