topic Re: Splunk Time stamp Error in Getting Data In

Splunk Time stamp Error

clyde772 — Tue, 29 Jun 2010 18:53:25 GMT

Hey Gurus!

I am processing F/W log such as below which I recieve through syslog server.

2010-06-29T20:48:26.742950+10:00 10.91.30.1 10110decf42010062920: 49:10Deny17090223182612192.168.100.1102410.91.50.43514eth0eth5````1971```

I am using rsyslogd daemon to process syslog packets, and somehow in the date field it looks like this,

2010-06-29T20:48:26

You realize that there is "T" in between the date and time. I don;t know if this is the cause but Splunk seems to process the date as 1 hour behind.

2010-06-29 07:48:26 PM in splunk.

Instead of

2010-06-29 08:48:26 PM

Have anybody seen similar problem? How can I fix it?

YhC.

Re: Splunk Time stamp Error

Lowell — Tue, 29 Jun 2010 22:15:50 GMT

Are you sure this isn't a timezone issue? Are you +10? if you could post the value of date_zone field for your indexed events, that would help.

Re: Splunk Time stamp Error

Lowell — Tue, 29 Jun 2010 22:17:52 GMT

Does your event literally contain a bunch of backticks ("`"), or is that for formatting your event? (Try indenting your sample by 4 spaces if you want to capture a literal string.

Re: Splunk Time stamp Error

Lowell — Tue, 29 Jun 2010 22:46:34 GMT

What's your props.conf entry for this source/sourcetype? Do you have a TIME_FORMAT specified, or is splunk automatically extracting timestamps?

Re: Splunk Time stamp Error

hiwell — Tue, 29 Jun 2010 23:12:07 GMT

To answer Lowell's question I believe Splunk is extracting the time automatically. If he is specifying the TIME_FORMAT the adding of 1hour can be avoided easily I believe - by just writing the regular expression

You could extract the time manually by adding a few things in the props.conf file. Something like

[<sourcetype>]
TIME_FORMAT=%Y-%m-%dT%H:%M:%S

I just did the same thing yesterday because Splunk wasn't extracting the timestamps correctly

more info here http://www.splunk.com/base/Documentation/3.0.2/Admin/Propsconf
and here about the time formatting http://apidock.com/ruby/DateTime/strftime

Re: Splunk Time stamp Error

Lowell — Tue, 29 Jun 2010 23:57:22 GMT

don't think its quite that simple. There is really nothing you can do with a regex to solve this problem; you can't add/subtract with a regex. Also, since the log file includes some timezone info, we should probably attempt to use it in TIME_FORMAT if at all possible, unless it's buggy. (BTW, the splunk docs you linked to (3.0.2) are very old at this point, 4.1.3 is the "latest" right now.)

Re: Splunk Time stamp Error

hiwell — Wed, 30 Jun 2010 01:19:38 GMT

Yeah you can't remove that T with it.

Re: Splunk Time stamp Error

Lowell — Wed, 30 Jun 2010 02:50:16 GMT

No, actually you can remove the "T" with a regex, but that kind of transformation happens after timestamp recognition so you would have keep the "T" in TIME_FORMAT.... But my point was that you can't modify the timestamp with a regex prior to the timestamping process and even if you could, you can't use a regex to do arithmetic; thus you can't change the time +/- an hour.

Re: Splunk Time stamp Error

Lowell — Wed, 30 Jun 2010 03:00:03 GMT

If you haven't tried using an explicit TIME_FORMAT in your props.conf file for this sourcetype, then I would suggest that you try the following:

TIME_PREFIX = ^
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%6N%z

If this doesn't work for you, then you may need to hardcode a timezone using the TZ setting in your props.conf file. If you end up using a custom timezone, then you may need to remove the %z from the above, I'm not sure how that works exactly. Another consideration is this: With the daylight savings changes (aka timezone changes) in the last couple of years, it's possible that whatever app that is creating this log has an old timezone table that could be out of date--just another possibility to consider.

Docs:

Specify timezones of timestamps

Re: Splunk Time stamp Error

hiwell — Wed, 30 Jun 2010 03:19:01 GMT

Hah you probably know whats going on but you're getting me confused here.. I meant to say that you have to keep the T in timestamp if you use the TIME_FORMAT I provided in the above comment. Is this what you're trying to say.

Re: Splunk Time stamp Error

mitch — Wed, 28 Jul 2010 11:26:00 GMT

I agree with Lowell above -- if it's exactly an hour off it's probably not a time extraction issue, but a timezone issue.
Info that would be useful:
* Do a search that finds that single event and then add "| eval t=_time | table t" to the search. You should see the raw UNIX epoch time (i.e. seconds since Midnight 1/1/1970, GMT). What number do we report that for that event. It'll be a number around 1.2 billion.
* What OS are you running?
* What version of Splunk?
* What timezone are you in? How specifically is it configured in the OS?

Re: Splunk Time stamp Error

blebit — Fri, 24 Jan 2014 08:54:41 GMT

hello, can we push this from Deployment Monitor ??