Splunk is getting confused Australian Timestamps thinking its US when Days (DD) are from 01 to 10 (Splunk6)

[Dark_Ichigo]

**Please Note:* This works fine with Splunk V4 but not Splunk V6.*

If the day of the month is below the 10th digit as the DD of a timestamp splunk will assume the date to be in US format (MM/DD/YYYY), otherwise it will correctly index the date as per the props Config as DD/MM/YYYY eg:

When splunk indexes the following log, it’s being indexed as being dated 10th of September

09/10/2013 04:30:28.115036 TRACE m 913   41    "[P00861257] FROM_LOCKER LCK;OW="hhhhhh,M,41";RE=OK;LO="xxxxxxxx""
09/10/2013 04:30:28.115569 TRACE m 913   41    "[P00861257] TO_EXCHANGE:MSDVXX=xxxxxx"
09/10/2013 04:30:28.156988 TRACE d 950   1     "[P00861257] COMMS_QUEUING:MSDVXX: RECV='08/10/2013 04:30:28.116353' QUEUE_TIME='         ' TO_DEV='08/10/2013 04:30:28.116878' FROM_DEV='08/10/2013 04:30:28.156627' DEV_TIME='00000.039'"

When splunk indexes the following log, it’s being indexed as being dated 13th of October (as per Config in props.conf)

13/10/2013 04:30:28.115036 TRACE m 913   41    "[P00861257] FROM_LOCKER LCK;OW="hhhhhh,M,41";RE=OK;LO="xxxxxxxx""
13/10/2013 04:30:28.115569 TRACE m 913   41    "[P00861257] TO_EXCHANGE:MSDVXX=xxxxxx"
13/10/2013 04:30:28.156988 TRACE d 950   1     "[P00861257] COMMS_QUEUING:MSDVXX: RECV='08/10/2013 04:30:28.116353' QUEUE_TIME='         ' TO_DEV='08/10/2013 04:30:28.116878' FROM_DEV='08/10/2013 04:30:28.156627' DEV_TIME='00000.039'"

######################################

Indexes.conf

 [index_name]
    homePath = $SPLUNK_DB/index_name/db
    coldPath = $SPLUNK_DB/index_name/colddb
    thawedPath = $SPLUNK_DB/index_name/thaweddb
    disabled = 0
    frozenTimePeriodInSecs = 2147483647

#####################################

props.conf

[source2]
MAX_EVENTS = 30000
TRUNCATE = 0
TIME_FORMAT=%d/%m/%Y %H:%M:%S.%6N
MAX_TIMESTAMP_LOOKAHEAD = 20
TIME_PREFIX = ^
TZ = Australia/Victoria

######################################

Forwarder Config (inputs.conf)

[default]
host = ABC  
[monitor://home/file/location/]
index = index
sourcetype = source2
crcSalt = <SOURCE>
disabled = false

[mchang_splunk · Accepted solution]

The root cause should be MAX_TIMESTAMP_LOOKAHEAD = 20.
The total length of the time stamp in the event is 26, to identify microsecond correctly, you should set:
MAX_TIMESTAMP_LOOKAHEAD = 26
In this way, this issue should be resolved in 6.0 and the correct microsecond will be shown in 4.x and 5.x.

[Dark_Ichigo]

We deployed it and it worked fine.

Cheers!

[mchang_splunk]

Please try having props.conf on the forwarder, I guess you have installed the heavy weight forwarder, not universal forwarder.

[Dark_Ichigo]

I'm using a Universal forwarder