topic Re: Splunk is getting confused Australian Timestamps thinking its US when Days (DD) are from 01 to 10 (Splunk6) in Getting Data In

Splunk is getting confused Australian Timestamps thinking its US when Days (DD) are from 01 to 10 (Splunk6)

Dark_Ichigo — Thu, 24 Oct 2013 00:43:23 GMT

**Please Note:* This works fine with Splunk V4 but not Splunk V6.*

If the day of the month is below the 10th digit as the DD of a timestamp splunk will assume the date to be in US format (MM/DD/YYYY), otherwise it will correctly index the date as per the props Config as DD/MM/YYYY eg:

When splunk indexes the following log, it’s being indexed as being dated 10th of September

09/10/2013 04:30:28.115036 TRACE m 913   41    "[P00861257] FROM_LOCKER LCK;OW="hhhhhh,M,41";RE=OK;LO="xxxxxxxx""
09/10/2013 04:30:28.115569 TRACE m 913   41    "[P00861257] TO_EXCHANGE:MSDVXX=xxxxxx"
09/10/2013 04:30:28.156988 TRACE d 950   1     "[P00861257] COMMS_QUEUING:MSDVXX: RECV='08/10/2013 04:30:28.116353' QUEUE_TIME='         ' TO_DEV='08/10/2013 04:30:28.116878' FROM_DEV='08/10/2013 04:30:28.156627' DEV_TIME='00000.039'"

When splunk indexes the following log, it’s being indexed as being dated 13th of October (as per Config in props.conf)

13/10/2013 04:30:28.115036 TRACE m 913   41    "[P00861257] FROM_LOCKER LCK;OW="hhhhhh,M,41";RE=OK;LO="xxxxxxxx""
13/10/2013 04:30:28.115569 TRACE m 913   41    "[P00861257] TO_EXCHANGE:MSDVXX=xxxxxx"
13/10/2013 04:30:28.156988 TRACE d 950   1     "[P00861257] COMMS_QUEUING:MSDVXX: RECV='08/10/2013 04:30:28.116353' QUEUE_TIME='         ' TO_DEV='08/10/2013 04:30:28.116878' FROM_DEV='08/10/2013 04:30:28.156627' DEV_TIME='00000.039'"

######################################

Indexes.conf

 [index_name]
    homePath = $SPLUNK_DB/index_name/db
    coldPath = $SPLUNK_DB/index_name/colddb
    thawedPath = $SPLUNK_DB/index_name/thaweddb
    disabled = 0
    frozenTimePeriodInSecs = 2147483647

#####################################

props.conf

[source2]
MAX_EVENTS = 30000
TRUNCATE = 0
TIME_FORMAT=%d/%m/%Y %H:%M:%S.%6N
MAX_TIMESTAMP_LOOKAHEAD = 20
TIME_PREFIX = ^
TZ = Australia/Victoria

######################################

Forwarder Config (inputs.conf)

[default]
host = ABC  
[monitor://home/file/location/]
index = index
sourcetype = source2
crcSalt = <SOURCE>
disabled = false

Re: Splunk is getting confused Australian Timestamps thinking its US when Days (DD) are from 01 to 10 (Splunk6)

mchang_splunk — Fri, 25 Oct 2013 07:19:57 GMT

Please try having props.conf on the forwarder, I guess you have installed the heavy weight forwarder, not universal forwarder.

Re: Splunk is getting confused Australian Timestamps thinking its US when Days (DD) are from 01 to 10 (Splunk6)

Dark_Ichigo — Fri, 25 Oct 2013 07:25:00 GMT

I'm using a Universal forwarder

Re: Splunk is getting confused Australian Timestamps thinking its US when Days (DD) are from 01 to 10 (Splunk6)

mchang_splunk — Mon, 28 Sep 2020 15:05:11 GMT

The root cause should be MAX_TIMESTAMP_LOOKAHEAD = 20.
The total length of the time stamp in the event is 26, to identify microsecond correctly, you should set:
MAX_TIMESTAMP_LOOKAHEAD = 26
In this way, this issue should be resolved in 6.0 and the correct microsecond will be shown in 4.x and 5.x.

Re: Splunk is getting confused Australian Timestamps thinking its US when Days (DD) are from 01 to 10 (Splunk6)

Dark_Ichigo — Thu, 05 Dec 2013 23:22:35 GMT

We deployed it and it worked fine.

Cheers!