I am processing F/W log such as below which I recieve through syslog server.
2010-06-29T20:48:26.742950+10:00 10.91.30.1 1
I am using rsyslogd daemon to process syslog packets, and somehow in the date field it looks like this,
You realize that there is "T" in between the date and time. I don;t know if this is the cause but Splunk seems to process the date as 1 hour behind.
2010-06-29 07:48:26 PM in splunk.
2010-06-29 08:48:26 PM
Have anybody seen similar problem? How can I fix it?
I agree with Lowell above -- if it's exactly an hour off it's probably not a time extraction issue, but a timezone issue.
Info that would be useful:
* Do a search that finds that single event and then add "| eval t=_time | table t" to the search. You should see the raw UNIX epoch time (i.e. seconds since Midnight 1/1/1970, GMT). What number do we report that for that event. It'll be a number around 1.2 billion.
* What OS are you running?
* What version of Splunk?
* What timezone are you in? How specifically is it configured in the OS?
If you haven't tried using an explicit
TIME_FORMAT in your
props.conf file for this sourcetype, then I would suggest that you try the following:
TIME_PREFIX = ^ TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%6N%z
If this doesn't work for you, then you may need to hardcode a timezone using the
TZ setting in your
props.conf file. If you end up using a custom timezone, then you may need to remove the
%z from the above, I'm not sure how that works exactly. Another consideration is this: With the daylight savings changes (aka timezone changes) in the last couple of years, it's possible that whatever app that is creating this log has an old timezone table that could be out of date--just another possibility to consider.
To answer Lowell's question I believe Splunk is extracting the time automatically. If he is specifying the TIME_FORMAT the adding of 1hour can be avoided easily I believe - by just writing the regular expression
You could extract the time manually by adding a few things in the props.conf file. Something like
I just did the same thing yesterday because Splunk wasn't extracting the timestamps correctly
more info here
and here about the time formatting http://apidock.com/ruby/DateTime/strftime
Hah you probably know whats going on but you're getting me confused here.. I meant to say that you have to keep the T in timestamp if you use the TIME_FORMAT I provided in the above comment. Is this what you're trying to say.
No, actually you can remove the "T" with a regex, but that kind of transformation happens after timestamp recognition so you would have keep the "T" in
TIME_FORMAT.... But my point was that you can't modify the timestamp with a regex prior to the timestamping process and even if you could, you can't use a regex to do arithmetic; thus you can't change the time +/- an hour.
don't think its quite that simple. There is really nothing you can do with a regex to solve this problem; you can't add/subtract with a regex. Also, since the log file includes some timezone info, we should probably attempt to use it in
TIME_FORMAT if at all possible, unless it's buggy. (BTW, the splunk docs you linked to (3.0.2) are very old at this point, 4.1.3 is the "latest" right now.)