Getting Data In

Search for events that have not happened

Engager

When we 'fall back' one hour for Daylight Savings Time, I'd like to run a search that would reveal those log sources that didn't adjust correctly (their event timestamps will be one hour in the future.) What might the syntax for such a search look like? Thank you!

Tags (1)
2 Solutions

SplunkTrust
SplunkTrust

The way I've usually seen this done is to compare _time to _indextime.

_time is the time extracted from the event's text, as the number of seconds since 1970

_indextime is the time the event was indexed, according to the local system clock (also number of seconds since 1970).

so if you use eval to look for this difference you should have what you need.

View solution in original post

Splunk Employee
Splunk Employee

Ah, the nefarious "future event".

For a lightweight approach, how about:

| metadata type=hosts | eval now = now() | where latestTime > now

View solution in original post

0 Karma

Path Finder

How well does the search "| metadata type=hosts | eval now = now() | where latestTime > now" work in a scheduled search? I seem to get a huge load of results for hosts off by as much as 2-3 minutes because of what - now() evaluates to when the search is scheduled to run? If I run the search manually, e.g. over the past hour, it seems to return accurate results.

0 Karma

Splunk Employee
Splunk Employee

Ah, the nefarious "future event".

For a lightweight approach, how about:

| metadata type=hosts | eval now = now() | where latestTime > now

View solution in original post

0 Karma

SplunkTrust
SplunkTrust

The way I've usually seen this done is to compare _time to _indextime.

_time is the time extracted from the event's text, as the number of seconds since 1970

_indextime is the time the event was indexed, according to the local system clock (also number of seconds since 1970).

so if you use eval to look for this difference you should have what you need.

View solution in original post

SplunkTrust
SplunkTrust

Well more generally you want to see anywhere where there's a delta between _indexTime and _time If both sides respect DST there's no issue. If only one respects DST there'll be a persistent delta suddenly, and if they're slightly off there'll be a delta for some small slice of time and then it'll go away. And in each case which direction the delta goes depends on which side is observing DST and which is not.

0 Karma

Splunk Employee
Splunk Employee

This assumes that your Splunk servers' clocks are affected by DST 🙂

0 Karma