topic Re: what is apiStartTime='ZERO_TIME' in Splunk Search

what is apiStartTime='ZERO_TIME'

sansay — Mon, 28 Sep 2020 14:10:19 GMT

I have been investigating excessively expensive searches by querying the audit log, and I came across one that has this time range:
apiStartTime='ZERO_TIME', apiEndTime='ZERO_TIME'

Anyone knows what this means?

Re: what is apiStartTime='ZERO_TIME'

lguinn2 — Tue, 25 Jun 2013 23:39:14 GMT

The audit log captures the time range of the search. As a Splunk user, you specify the time range by using the pull-down menu (or by using the earliest and latest keywords). When Splunk processes the search, it calculates the actual time that should be searched. apiStartTime represents the earliest time, and apiEndTime represents the latest time.

EDIT - in my original answer, I said

apiStartTime='ZERO_TIME', apiEndTime='ZERO_TIME' means that the search ran over All Time. It makes sense that this would be an excessively expensive search.

but this appears not to be the case.

END EDIT

Re: what is apiStartTime='ZERO_TIME'

sansay — Wed, 26 Jun 2013 00:10:21 GMT

Thank you very much lguinn.
The weird thing is that I disabled the "All time" from the GUI. And the user, from being the previous Splunk admin knows very well not to run "All time" queries. And he confirmed that when asked. So how else could this happen?

Is there any way I can get the exact query that was executed, ie, with the time range specified by the user?

Re: what is apiStartTime='ZERO_TIME'

sansay — Mon, 28 Sep 2020 14:10:22 GMT

This gets weirder and weirder, according to my last search, and if apiStartTime='ZERO_TIME', apiEndTime='ZERO_TIME' means "All time", even I ran "All time" queries. This is starting to sound more and more like a bug.

Re: what is apiStartTime='ZERO_TIME'

lguinn2 — Wed, 26 Jun 2013 00:58:49 GMT

Perhaps I am wrong. Could this have been something run by Splunk internally?

Re: what is apiStartTime='ZERO_TIME'

sansay — Mon, 28 Sep 2020 14:10:53 GMT

Sorry but, indeed, it seems that your original answer is wrong.
A simpler search, without apiStartTime='ZERO_TIME' apiEndTime='ZERO_TIME', returns a bunch of other records, including the very same query, with the exact time range selected by the user. And this query occured just microseconds before the one with ZERO_TIME. So it must be something splunk does, but because it happens all the time it can't mean that it's the "All time" time range that was used.
So I have to remove the point. I will add this in a splunk ticket I opened to resolve cold storage searches that take our system down.

Re: what is apiStartTime='ZERO_TIME'

lguinn2 — Thu, 27 Jun 2013 01:15:50 GMT

No problem - please post if you figure it out...

Re: what is apiStartTime='ZERO_TIME'

sarnagar — Tue, 29 Sep 2020 13:07:27 GMT

@sansay ,
Could you please let me know wht this actually means if you are aware of it now?

apiStartTime='ZERO_TIME', apiEndTime='ZERO_TIME'

Re: what is apiStartTime='ZERO_TIME'

sansay — Wed, 08 Mar 2017 23:13:54 GMT

Sorry but no, I haven't figured it out. I haven't had the time to even think about this issue.

Re: what is apiStartTime='ZERO_TIME'

dvg06 — Tue, 29 Sep 2020 18:32:35 GMT

These could be real time searches.
I ran a search like "index=*" for 30 seconds realtime, and the apiStartTime was displayed as Zero_time

search total_run_time _time apiStartTime apiEndTime search_type user
search index=* 2018-03-20 10:28:09.913 ZERO_TIME ZERO_TIME ad hoc test_user01
search index=* 2018-03-20 10:28:13.560 ZERO_TIME ZERO_TIME ad hoc test_user01