Knowledge Management
Highlighted

Summary Index Backfiller fill_summary_index.py broken? "No scheduled times for your time range"

Path Finder

Hi Fellow Splunkers,

After having upgraded to 6.4.1 yesterday, I had a go with fillsummaryindex.py again, and noticed that am still getting the same error "No scheduled times for your time range", which I had been getting since a year or longer.

Being in dire need of it this time, I tried to backfill different searches, e.g. using sistats, using collect, basic and cron scheduling. To my surprise even the most basic search (basic scheduling 1 hour, using sistats and flagging "enable summary indexing" in the webgui search edit form), was getting the same error "No scheduled times...".

Taking a deeper look at it with @daubsi_2, we found that:

  • manually searching the designated REST endpoint (saved/searches/{name}/scheduledtimes) returned the needed list "scheduledtimes"

      <s:list>
        <s:item>1465292100</s:item>
        <s:item>1465293000</s:item>
        ...
    
  • Looking at the function output of getSavedSearchWithTimes in fillsummaryindex.py, this returned the definition of the search in question, but without scheduledtimes. It looks pretty much as if only search definition without scheduledtimes was queried

  • Taking a look a saved.py and entity.py, we could not make out where the needed REST URI .../scheduled_times would be prepended

  • Changing a few lines to make fillsummaryindex.py use the REST endpoint (saved/searches/{name}/scheduledtimes) made the script run again as it used to. It found the scheduledtimes just happily.

How is your mileage with fillsummaryindex.py? Does it work as intended in 6.3+? As it is probably in heavy use out there, I would be a little surprised if it's really going to the wrong endpoint.

Olli

Highlighted

Re: Summary Index Backfiller fill_summary_index.py broken? "No scheduled times for your time range"

Path Finder

Here is a quick hack to fix this - if anyone needs it, too. Beware of mistakes, use at you own risk, I am no pro at that.

First, check if you are getting the same error:
- Is your search configured correctly (enabled, isscheduled, has summaryindex action)?
- Check if the REST Endpoint returns the scheduled times correctly, when queried manually:

curl -k -u <user>:<pass> https://localhost:8089/servicesNS/<user>/<app>/saved/searches/<searchname>/scheduled_times --get -d earliest_time=-5d -d latest_time=now

This REST query should return a block with the scheduled_times you queried for (-5d until now, as set above), e.g.:

<s:key name="scheduled_times">
          <s:list>
            <s:item>1465548600</s:item>
            <s:item>1465549500</s:item>
            <s:item>1465550400</s:item>
            ....
          </s:list>
</s:key>

If it does not return this list of scheduledtimes of jobs, there is something else wrong. This hack won't help you until you get schedulestimes.

To integrate this hack, make the following changes to fillsummaryindex.py:
Within the imports at the beginning add the following line, after line 11 "import splunk.util" would be a nice place:

import splunk.entity as entity

Comment out line 352 like this:

#ent = splunk.search.getSavedSearchWithTimes(ssname, et, lt, namespace=app, sessionKey=sk, owner=owner)

Insert this replacement statement in the following line 353:

ent = entity.getEntity("/saved/searches", ssname, uri="/servicesNS/"+owner+"/"+app+"/saved/searches/"+ssname+"/scheduled_times", earliest_time=et, latest_time=lt, namespace=app, owner=owner, sessionKey=sk)

What this does is that it leaves out the call to the wrapper function getSavedSearchWithTimes in saved.py and calls the getEntity function from entity.py directly, with an override to the correct REST Endpoint (in field "uri").

As said, use at own risk. If anyone has a nicer solution, feel free to post here.

Please upvote of you have the same problem and this helped you.

Highlighted

Re: Summary Index Backfiller fill_summary_index.py broken? "No scheduled times for your time range"

Engager

For your convenience, here's a diff so you can apply the patch:

--

--- fill_summary_index.py       2016-05-13 00:05:14.000000000 +0200
+++ fill_summary_index_oko3.py  2016-06-10 15:17:18.000000000 +0200
@@ -9,6 +9,7 @@
 import splunk.auth
 import splunk.search
 import splunk.util
+import splunk.entity as entity
 import splunk.clilib.bundle_paths as bundle_paths

 app = None
@@ -158,12 +159,12 @@
       -timefield <string>   The field in summary index data that contains the
                             scheduled time of the saved search that generated
                             that data
-
+
   1 - For a full description of Splunk relative time strings, see:
       http://docs.splunk.com/Documentation/Splunk/6.4.1/User/ChangeTheTimeRangeOfYourSearch#Syntax_for_rel...
   '''
   sys.exit(0)
-
+
 def getBoolArg(opt, val):
   try:
     return splunk.util.normalizeBoolean(val, enableStrictMode=True)
@@ -349,7 +350,8 @@
     cur_st_list = []
     index = indexarg
     try:
-      ent = splunk.search.getSavedSearchWithTimes(ssname, et, lt, namespace=app, sessionKey=sk, owner=owner)
+      #ent = splunk.search.getSavedSearchWithTimes(ssname, et, lt, namespace=app, sessionKey=sk, owner=owner)
+      ent = entity.getEntity("/saved/searches", ssname, uri="/servicesNS/"+owner+"/"+app+"/saved/searches/"+ssname+"/scheduled_times", earliest_time=et, latest_time=lt, namespace=app, owner=owner, sessionKey=sk)
       if 'scheduled_times' in ent:
         for st in ent['scheduled_times']:
           cur_st_list.append((ssname, st))

View solution in original post