Splunk Search

what is apiStartTime='ZERO_TIME'

Contributor

I have been investigating excessively expensive searches by querying the audit log, and I came across one that has this time range:
apiStartTime='ZEROTIME', apiEndTime='ZEROTIME'

Anyone knows what this means?

Tags (3)

Path Finder

These could be real time searches.
I ran a search like "index=*" for 30 seconds realtime, and the apiStartTime was displayed as Zero_time

search totalruntime time apiStartTime apiEndTime searchtype user
search index=* 2018-03-20 10:28:09.913 ZEROTIME ZEROTIME ad hoc testuser01
search index=* 2018-03-20 10:28:13.560 ZERO
TIME ZEROTIME ad hoc testuser01

0 Karma

Legend

The audit log captures the time range of the search. As a Splunk user, you specify the time range by using the pull-down menu (or by using the earliest and latest keywords). When Splunk processes the search, it calculates the actual time that should be searched. apiStartTime represents the earliest time, and apiEndTime represents the latest time.

EDIT - in my original answer, I said

apiStartTime='ZERO_TIME', apiEndTime='ZERO_TIME' means that the search ran over All Time. It makes sense that this would be an excessively expensive search.

but this appears not to be the case.

END EDIT

0 Karma

Legend

No problem - please post if you figure it out...

0 Karma

Contributor

Sorry but, indeed, it seems that your original answer is wrong.
A simpler search, without apiStartTime='ZEROTIME' apiEndTime='ZEROTIME', returns a bunch of other records, including the very same query, with the exact time range selected by the user. And this query occured just microseconds before the one with ZERO_TIME. So it must be something splunk does, but because it happens all the time it can't mean that it's the "All time" time range that was used.
So I have to remove the point. I will add this in a splunk ticket I opened to resolve cold storage searches that take our system down.

0 Karma

Contributor

@sansay ,
Could you please let me know wht this actually means if you are aware of it now?

apiStartTime='ZEROTIME', apiEndTime='ZEROTIME'

0 Karma

Contributor

Sorry but no, I haven't figured it out. I haven't had the time to even think about this issue.

0 Karma

Legend

Perhaps I am wrong. Could this have been something run by Splunk internally?

0 Karma

Contributor

This gets weirder and weirder, according to my last search, and if apiStartTime='ZEROTIME', apiEndTime='ZEROTIME' means "All time", even I ran "All time" queries. This is starting to sound more and more like a bug.

0 Karma

Contributor

Thank you very much lguinn.
The weird thing is that I disabled the "All time" from the GUI. And the user, from being the previous Splunk admin knows very well not to run "All time" queries. And he confirmed that when asked. So how else could this happen?

Is there any way I can get the exact query that was executed, ie, with the time range specified by the user?

0 Karma