We have an indexer indexing events with _time 5 hours head and we have Distributed search from SH which looks at _index time earliest and latest 10 minutes...although events with _time + 5 hours and matching index time exist ..they dont show up in Splunk SH scheduled searches ? why ?
Does the scheduler (SH) introduce some filter when they run to prevent them from searching events that have timestamps later than the local runtimes of the queries
Kindly clarify
Yeah, but _index_earliest
and _index_latest
are added to the time range as AND
-condition, they don't replace it. Here's an example:
I've just started the Splunk on my Laptop, it's 9am... in the past hour it has indexed data from this morning, plus a bit of leftovers from yesterday. I'm running two searches based on this string:
index=_internal _index_earliest=-h _index_latest=now
If I run that with a time range (timestamp, not index time) of -24h@h to now I get events from yesterday that were indexed this morning, if I run that with a time range (timestamp, not index time) of -4h@m to now I don't get those.
To get around this for events from the future, set the time range's latest setting to something in the future, for example +2d
.
Yeah, but _index_earliest
and _index_latest
are added to the time range as AND
-condition, they don't replace it. Here's an example:
I've just started the Splunk on my Laptop, it's 9am... in the past hour it has indexed data from this morning, plus a bit of leftovers from yesterday. I'm running two searches based on this string:
index=_internal _index_earliest=-h _index_latest=now
If I run that with a time range (timestamp, not index time) of -24h@h to now I get events from yesterday that were indexed this morning, if I run that with a time range (timestamp, not index time) of -4h@m to now I don't get those.
To get around this for events from the future, set the time range's latest setting to something in the future, for example +2d
.
The time range never appears in the lispy debug messages.
Try choosing latest=+10y
to effectively disable the latest time range boundary - come back once you've tried that and report on your results.
..i have searched the search.log Lispy query search parser of this search for these _indextime "AND" all _time cannot seem to find it ...is the final query dispatched available to see in any logs..i keep seeing only indextime..
All filters are linked by AND
, especially the time range. Just set latest
to +10y
.
On a concept level if we have inline search time range of past 5 minutes of indextime and outside timerange of past 10 min..so will it AND and always give both last 5 indextime and last 10 _time..making our key purpose ineffective
I did not get the precedence or how do 2 time ranges operate ?
Use +10y
then, that's most likely beyond your MAX_DAYS_HENCE
setting anyway.
Is there anyway we can just search using _indextime and not use _time
Ok the idea is we dont know all device timezone and timestamps...how can we be sure of +2d ...it can be a year head ..irrespective we use indextime in seach query
If you specify _index_earliest
inline then you're not overriding any earliest
set elsewhere. Both filters can coexist and be applied.
As for "no _time
", every event in Splunk should have _time
, even if it's just derived from the index time.
If the timerange is blank then it should use "all time", which is earliest=0 latest=now
.
Have you tried what I suggested several times, setting latest as a time in the future, e.g. +2d
?
Also note i use scheduled search and timerange is blank excepting for inline search timerange based on _index_time
Hmm... inline search timerange is supposed to override Timerange specified outside ...i saw that documented somewhere ..also in our case the timerange outside our search is blank...ie we have no _time
I believe you can do _index_earliest and latest based search instead of _time and thats what was used
Isn't every search based on _time
, regardless of other filters?
I guess im trying to say the search query is based on _indextime and not _time...so that should not be the case although _time is +5 hrs ahead
If your earliest
and latest
filter for ten minutes then you won't see events a day into the future.
Distributed search from SH which looks at _index time earliest and latest 10 minutes..as in above ...thanks
What time range is the scheduled search using?