Alerting

Search to discover events indexed with incorrect TZ offset?

the_wolverine
Champion

I want to set up an alert for when we start receiving events that are > 30 minutes off the idxtime. This would indicate a possible issue with TZ offset (perhaps the server admin changed the TZ without informing the Splunk admin.)

Anyone have such a search available?

0 Karma
1 Solution

the_wolverine
Champion

I use the following query:

index=* | eval lag=_time-_indextime | search lag>1000 OR lag<-1000 | convert ctime(_indextime) as idxtime |  stats latest(_time) , latest(idxtime) , max(lag) as lag by index,host,source | rangemap field=lag EST="9000-10000" MST="2000-4000" CST="5000-7000" PST="-1000-1000" | rename range as TZ

I've approximated the ranges but it gets it close enough. This is more to catch hosts sending events in their local time without a TZ specified but it can also catch hosts that are not NTP-synced.

View solution in original post

the_wolverine
Champion

I use the following query:

index=* | eval lag=_time-_indextime | search lag>1000 OR lag<-1000 | convert ctime(_indextime) as idxtime |  stats latest(_time) , latest(idxtime) , max(lag) as lag by index,host,source | rangemap field=lag EST="9000-10000" MST="2000-4000" CST="5000-7000" PST="-1000-1000" | rename range as TZ

I've approximated the ranges but it gets it close enough. This is more to catch hosts sending events in their local time without a TZ specified but it can also catch hosts that are not NTP-synced.

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...