We had logs initially with timestamp: [05/18/15 6:00:02.3898 AM]
With the latest release, the timestamp in logs changed to [05/18/15 6:00:02.3898]. There is no local equivalent for time (AM or PM).
indexes.conf :
TIME_PREFIX = ^\[
TIME_FORMAT = %m/%d/%y %H:%M:%S.%4N %p
MAX_TIMESTAMP_LOOKAHEAD = 40
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\n\r]+)\[\d+\/\d+\/\d+\s+\d+\:\d+\:\d+\.\d+\s*\w*\]
TRUNCATE = 20000
KV_MODE = none
But we wanted to index a few more old logs along with new ones with new timestamp format. What can we modify in the indexes.conf for TIME_FORMAT, or should we use any other customization? This is becoming very expensive for us while indexing, as it slows down spending time parsing on these logs. And I feel the WARN messages I see in splunkd.log are due to the above change in log time stamp formats.
07-29-2015 13:42:27.889 -0700 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Sun Jul 26 23:48:00 2015).
New log examples :
[07/26/15 0:00:13.2596] ABC {067}: <67afc72a-4b5b-4e07-a458-eed2b7dc30b2>Abc.Efg("xyz") by jkin/xyz/7bb07297-ffcc-49ab-bb0e-5b9536ab3eb6. ,<0ms>
[07/26/15 0:00:13.3220] ABC {067}: <c8e592fa-df82-4247-88a0-2e819b7ca74c>Abc.Efg(77) by jking/xyz/7bb07297-ffcc-49ab-bb0e-5b9536ab3eb6. ,<406ms>
old Log examples :
[05/18/15 6:00:02.4210 AM] ABC {614}: Abc.Efg("0pk1") by 1ml3/abc/dbc910c1-0958-46d2-a497-e19c91e72408. ******** (766 MB)/35 Objects
Thanks
