How do I properly describe non-standard date/time format? Jdate (ordinal) year within century) and military time.

[jldebell]

I am working with a | delimited field log. The second column is the jdate and the third column appears to be a epoch time. The julian date is formatted as year, day of the year... 15 = 2015 and 085 = 3/26/15. I am guessing the time is formatted as hh:mm:ss.

field1|15085|232038|
field1|15085|231633|
field1|15085|203812|
field1|15085|203812|

props.conf is defined as the following:

FIELD_DELIMITER = |
FIELD_NAMES = type,jdate,time,.....
TIMESTAMP_FIELDS = jdate,time
TIME_FORMAT = %Y%m%d %H%M%S

The data is indexing into Splunk and then it will stop for a couple of days and restart. Looking at _internal index, the following error message is populated:

A possible timestamp match (Fri Jan  4 05:24:39 2008) is outside of the acceptable time window.

I know the time stamp format is off, but I am not sure how to adjust it. Looking for suggestions if anyone else experienced this.

Thanks, Jenn

[jldebell · Accepted solution]

Here is the solution that worked for us:

[sourcetype]

TIME_FORMAT = %y%j|%H%M%S
TIME_PREFIX=\w{2}\|
TZ=UTC
MAX_TIMESTAMP_LOOKAHEAD=12
disabled = false
NO_BINARY_CHECK = true

Thanks again for your help!

[rsennett_splunk]

Hi Jenn,

You are nearly there. The TIME_FORMAT is actually pretty critical here, so you'll want to fix that, first off. you've told Splunk that this 15085 is %Y%m%d "year month day" and it isn't... it's the "year within a century": %y and the "day of the number of the year" : %j which is more specifically the "ordinal" date in the gregorian calendar... despite the "j"... the field you call "time" is just military time. Epoch time is something altogether different and can't be represented by the "time" part alone... (epoch time is the number of seconds that have elapsed since 00:00:00 UTC Thursday, 1 January 1970)

So TIME_FORMAT= %y%J%H%M%S

Should get you what you want...

[jldebell]

Thanks again for your help! This got us closer to solving the problem.

[rsennett_splunk]

That's great. Glad you got it worked out!