Getting Data In

Why isn't my TIME_FORMAT working for this syslog data?

flakshack
Explorer

I just configured a new device to send data to a syslog server (w/universal forwarder), but when it shows up in Splunk, the time is incorrect.  I have about 30 other devices from different vendors in the same configuration that are working fine.

Here's an example syslog entry:

2021-02-26T15:35:09-05:00 XYZ---Office-HQ edge[9076]: EDGE_NEW_DEVICE: New or updated client device b4:56:e3:a8:91:b5, ip 10.5.38.0

When this log entry shows up in Splunk, the _time is 3:35:09 PM (future) when it should be 10:35:09 AM.  The Splunk server (single-node) and device are both in the same time zone with me and other devices on the same syslog server are working fine.

I've reviewed the following posts, but haven't had much luck

How time zones are processed by Splunk

Configure timestamp recognition

props.conf documentation

For example, I set the sourcetype to "velocloud:syslog" for the input and I tried editing the sourcetype so that the TIME_FORMAT=%Y-%m-%dT%H:%M:%S%:z

Unfortunately, this hasn't had any effect. 

Snag_100037a7.png

I feel like I'm missing something simple, but I've now spent hours going through everything twice with no luck.  Any help would be appreciated.

 

Labels (1)
Tags (3)
0 Karma
1 Solution

gbeatty
Path Finder

I would try explicitly declaring your time zones, as it seems like the time lines up too perfectly to be anything else.  Is your timezone UTC -5 perhaps? (or perhaps UTC+5?)   From my experience time zones  between the time settings of the log source, sourcetype, forwarder, IDX/SH, and user settings the time zones can get quite messy.   There's been more than once I assumed everything was properly configured for UTC only to find an error somewhere.

View solution in original post

0 Karma

gbeatty
Path Finder

I would try explicitly declaring your time zones, as it seems like the time lines up too perfectly to be anything else.  Is your timezone UTC -5 perhaps? (or perhaps UTC+5?)   From my experience time zones  between the time settings of the log source, sourcetype, forwarder, IDX/SH, and user settings the time zones can get quite messy.   There's been more than once I assumed everything was properly configured for UTC only to find an error somewhere.

0 Karma

flakshack
Explorer

Thanks for the reply.  All of my systems are located in the same time zone (GMT-5) and I have that set in my user preferences in the Splunk UI.

Per your suggestion, I tried setting the time zone for the sourcetype (via the UI) to +5 and also later to -5.  Weirdly, neither setting made any difference in the _time of the log entries in Splunk.  In both cases, the time still showed up in the future.

So I decided to change the timestamp format to exclude the -05:00 and also change the time zone and that worked.

Timestamp format:  %Y-%m-%dT%H:%M:%S

Time Zone:  GMT

flakshack_0-1614960004387.png

Thanks for the help!

 

 

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...