topic Re: Time Stamps 1 day early? in Getting Data In

Time Stamps 1 day early?

empath — Sat, 17 Jul 2010 08:42:56 GMT

I've got a log file which tracks some call statistics.

For some reason, about half of these, Splunk has them as being exactly 1 day older than they are.

This one is indexed as 7/15/10

VQM: 8885551212 07/16/10.16:59:59 0.000% 0ms - G726 20ms 1 1

This one is indexed as 07/16/10

VQM: 8885551212 07/16/10.16:33:40 0.000% 0ms - G729 20ms 1 1

For the life of me, I cannot tell the difference between the two.

I found a field called timestartpos, but that just (correctly) shows the first character of the time. Is there something like that for date?

I also correctly set the Time Zone in props.conf and created a timeformat string for that source, which didn't make any difference.

Thanks for your assistance.

Re: Time Stamps 1 day early?

empath — Sat, 17 Jul 2010 09:06:54 GMT

Additional notes:

I ran splunk train date on this data, and for some reaosn it can't seem to find the date:

From: VQM: 8885551212 07/16/10.16:03:59 0.000% 0ms - G726 20ms 1 1
Parsed: Fri Jul 16 16:03:59 2010
UTC Time: 1279310639
Time Region: 25-34
Date Region: -1--1
From: VQM: 8885551212 07/16/10.16:03:59 0.000% 0ms - G726 20ms 1 1
Parsed: Fri Jul 16 16:03:59 2010
UTC Time: 1279310639
Time Region: 25-34
Date Region: -1--1

I assume Date Region 1 1 means that it couldn't find it?

Re: Time Stamps 1 day early?

empath — Sat, 17 Jul 2010 09:49:25 GMT

So my original questions is somewhat misleading. My question is now: Why isn't Splunk recognizing the date?

Re: Time Stamps 1 day early?

Lowell — Sat, 17 Jul 2010 09:55:13 GMT

I'm pretty sure timestartpos and timeendpos show the start/end position of the entire timestamp not just the time portion of it. (Although, I don't remember every verifying that, so it could just be an assumption.)

My guess would be that splunk is detecting some part of your date as a different timezone. You can look at the field date_zone and see if it is different between your indexed events.

As far as your props.conf entries, just to confirm you've tried something like this:

 [my_source_type]
 TIME_PREFIX = ^\S+ \d+
 TIME_FORMAT = %m/%d/%y.%H:%M:%S
 SHOULD_LINEMERGE = False

BTW, I'm not sure about the output of splunk date train stuff, the formatting is hard to follow in your comment, and to be honest, I've never found that tool all that useful. Setting an explicit TIME_FORMAT has always been easier and less awkward IMHO.

Re: Time Stamps 1 day early?

empath — Sat, 17 Jul 2010 10:39:05 GMT

Thanks, I had done Time_format, but I hadn't added the time prefix line! thanks.