accessing starting and ending time of a scheduled search inside the search

[guilhem]

I would like to access to the starting and ending time of a scheduled search inside the search itself, if I want to run it afterwards with the right parameters.

If I put the | addinfo command inside the scheduled search it gives me the info_min_time and info_max_time but these contains the time when the search was ran, not the time the search was scheduled to run.

In fact I want to access this time (when run with the fill_summary_index.py script):

Started job 'XXXX_cHJvZF9zdW1tYXJ5X3JldHVybmluZ19zZWdtZW50X2xhc3RIb3Vy_at_1363205100_2ca7751a68b1bed6' for saved search 'prod_summary_returning_segment_lastHour', UTC = 1363205100 (Wed Mar 13 21:05:00 2013)

Thanks!

---

EDIT:

"For saved search 'saved_search_name'
Executing search to find existing data: 'search splunk_server=local index=my_index source="saved_search_name" | stats count by search_now'
  waiting for job sid = '1363940863.11827'  ... finished
Out of 10 scheduled times, 3 will be skipped because they already exist.

*** Spawning a total of 7 searches (max 8 concurrent) ***
Started job 'XXXXcHJvZF9zdW1tYXJ5X3JldHVybmluZ19zZWdtZW50X2xhc3RIb3Vy_at_1363259100_86ab6683906837c8' for saved search 'prod_summary_returning_segment_lastHour', UTC = 1363259100 (Thu Mar 14 12:05:00 2013)

This is the time (UTC = ...) that I want to access, even if I run the search in an unscheduled manner, the real scheduled time is still computed by the python script somehow. I want to know if I can access this value inside the search itself?

Thanks for the answers!

Guilhem

[guilhem · Accepted solution]

So after messing around a bit, I have found a clunky way to achieve what I wanted, may it helps other!

First I have created a macro, that takes the last search of the jobs list with the given search name, and return its earliest time (MY_MACRO):

MY_MACRO(search_name, "hours to subtract before earliest schedule time")
[| rest /services/search/jobs
| search label="$arg1$"
| head 1
| eval earliest=substr(earliestTime, 1, 19)
| eval earliest=strptime(earliest,"%Y-%m-%dT%H:%M:%S")
| eval earliest=earliest-$arg2$*3600
| eval earliest=round(earliest, 0)
| return earliest]

Note that you shouldn't return $earliest as in a macro it will not work (maybe interpreted as an argument cause of the $ sign, see here.

Then, take the search you want to backfill summary index, let's take index=MY_INDEX as an example (name it MY_SEARCH), and add the MY_MACRO value at the end, so the final search is:

index=MY_INDEX MY_MACRO(MY_SEARCH, "hours to subtract before earliest schedule time")

You can then use fill_summary_index.py with this search, the earliest time will be the scheduled time (the UTC+ ... time you can see in the console log), and not the time you run the search.

Thanks very much to Ayn, Martin and alacercogitatus for the help, I think I'm done with it.

[Ayn]

You could get this from the REST endpoint /saved/searches, see more here: http://docs.splunk.com/Documentation/Splunk/5.0.2/RESTAPI/RESTsearch#saved.2Fsearches.2F.7Bname.7D

Specifically the next_scheduled_time value should be what you want.

[guilhem]

Yes I do, but I don't see all of my saved search inside. Only a very small portion of them, can't tell what is their common factor?

Same in the /services/scheduled/, I don't have any of my search here, just an empty view folder.

[martin_mueller]

You should have /services/saved/searches/ though.

[guilhem]

Thank you very much. I have tested the rest keyword and it works very weel. I was able to (almost) solve the problem. I still have a parsing error but it may be due to the macro I use (following topic here:
http://splunk-base.splunk.com/answers/80765/debugging-a-search-ran-from-python-script)

Also it looks like the endpoint you give ( saved/searches) doesn't exists in my system. I only have inputs in the /services/search.

[Ayn]

Note that you can access it using the rest command in the search language.

[guilhem]

I have never used this before (the REST API). Gonna need some time to test if I can make it work. Thanks for the answer.

[martin_mueller]

Wouldn't this be what now() returns?

[guilhem]

Yes, I see where is the confusion. But when you use the script fill_summary_index.py, in the console log, you can see (see edited post, not enough room here)

[Ayn]

I don't really get how you mean by scheduled here - if you run it manually on 3/1/2013 it's obviously not scheduled, so there's no way of accessing a scheduled time that you're not using anyway.

[guilhem]

The now() description is:

now(): This function takes no arguments and returns the time that the search was started. The time is represented in Unix time or seconds since epoch.

The 'search was started' is a bit confusing. Let's say this search was scheduled to run the 01/01/2013 and I run it the 03/01/2013, what will now() return: 01/01/2013 or 03/01/2013?