Getting Data In

Why isn't my TIME_FORMAT working for this syslog data?

flakshack
Explorer

I just configured a new device to send data to a syslog server (w/universal forwarder), but when it shows up in Splunk, the time is incorrect.  I have about 30 other devices from different vendors in the same configuration that are working fine.

Here's an example syslog entry:

2021-02-26T15:35:09-05:00 XYZ---Office-HQ edge[9076]: EDGE_NEW_DEVICE: New or updated client device b4:56:e3:a8:91:b5, ip 10.5.38.0

When this log entry shows up in Splunk, the _time is 3:35:09 PM (future) when it should be 10:35:09 AM.  The Splunk server (single-node) and device are both in the same time zone with me and other devices on the same syslog server are working fine.

I've reviewed the following posts, but haven't had much luck

How time zones are processed by Splunk

Configure timestamp recognition

props.conf documentation

For example, I set the sourcetype to "velocloud:syslog" for the input and I tried editing the sourcetype so that the TIME_FORMAT=%Y-%m-%dT%H:%M:%S%:z

Unfortunately, this hasn't had any effect. 

Snag_100037a7.png

I feel like I'm missing something simple, but I've now spent hours going through everything twice with no luck.  Any help would be appreciated.

 

Labels (1)
Tags (3)
0 Karma
1 Solution

gbeatty
Path Finder

I would try explicitly declaring your time zones, as it seems like the time lines up too perfectly to be anything else.  Is your timezone UTC -5 perhaps? (or perhaps UTC+5?)   From my experience time zones  between the time settings of the log source, sourcetype, forwarder, IDX/SH, and user settings the time zones can get quite messy.   There's been more than once I assumed everything was properly configured for UTC only to find an error somewhere.

View solution in original post

0 Karma

gbeatty
Path Finder

I would try explicitly declaring your time zones, as it seems like the time lines up too perfectly to be anything else.  Is your timezone UTC -5 perhaps? (or perhaps UTC+5?)   From my experience time zones  between the time settings of the log source, sourcetype, forwarder, IDX/SH, and user settings the time zones can get quite messy.   There's been more than once I assumed everything was properly configured for UTC only to find an error somewhere.

View solution in original post

0 Karma

flakshack
Explorer

Thanks for the reply.  All of my systems are located in the same time zone (GMT-5) and I have that set in my user preferences in the Splunk UI.

Per your suggestion, I tried setting the time zone for the sourcetype (via the UI) to +5 and also later to -5.  Weirdly, neither setting made any difference in the _time of the log entries in Splunk.  In both cases, the time still showed up in the future.

So I decided to change the timestamp format to exclude the -05:00 and also change the time zone and that worked.

Timestamp format:  %Y-%m-%dT%H:%M:%S

Time Zone:  GMT

flakshack_0-1614960004387.png

Thanks for the help!

 

 

0 Karma

Tune In & Win!

Don't miss out on your
chance to take home free
prizes by helping our players
save the Splunk Cloudom!

Dungeons & Data
Monsters: Splunk O11y
Day Editions Games
stream live:
5/4 at 6:30pm PST
5/5 at 7:00pm PST
on