Timestamp parsing - Failed to parse errors

[Derek]

Hi,

I have a log that contains large multiline events such as:

---- DS log entry made at 08/29/2011 11:03:00
*** Log is continued from intermediate LogID [15cc1510] ***
Message E:\FILES\SPOOL\DOMAINS\TEST.COM\B0391744043.RCP queued for remote delivery to domain TEST.COM (.LCK).
Message E:\FILES\SPOOL\DOMAINS\TEST.COM\B0391744045.RCP queued for remote delivery to domain TEST.COM (.LCK).
Message E:\FILES\SPOOL\DOMAINS\TEST.COM\B0391744047.RCP queued for remote delivery to domain TEST.COM (.LCK).
Message E:\FILES\SPOOL\DOMAINS\TEST.COM\B0391744051.RCP queued for remote delivery to domain TEST.COM (.LCK).
Message E:\FILES\SPOOL\DOMAINS\TEST.COM\B0391744053.RCP queued for remote delivery to domain TEST.COM (.LCK).
Message E:\FILES\SPOOL\DOMAINS\TEST.COM\B0391744054.RCP queued for remote delivery to domain TEST.COM (.LCK).
Message E:\FILES\SPOOL\DOMAINS\TEST.COM\B0391744056.RCP queued for remote delivery to domain TEST.COM (.LCK).
Message E:\FILES\SPOOL\DOMAINS\TEST.COM\B0391744062.RCP queued for remote delivery to domain TEST.COM (.LCK).
*** Intermediate LogID [15cc03e0] will be continued later. ***

I would like each line that starts with "Message" to be its own event so that I can search and manipulate related events. But the induvidual lines when I break them don't have a timestamp and so I get errors like:

DateParserVerbose - Failed to parse timestamp for event. Context="source::/opt/logs/OPR20110829-7.LOG|host::server1|mysourcetype|" Text="Message E:\FILES\SPOOL\DOMAINS\TEST.COM\B0391744043.RCP queued for remote delivery to domain"

Using props.conf of:

[mysourcetype]
LINE_BREAKER = ([\r\n]+)((----)|(***\s)|(Message\s))
SHOULD_LINEMERGE = false


When I look at the events in the search app they end up appearing in the correct order with a timestamp that based on precedence would seem like it's getting it from the previous events based on the documentation.

The source is /opt/logs/OPR20110829-7.LOG and I also tried a custom file name timestamp to with luck.

How can I supress the warnings in the splunkd logs if the timestamp precedence is working ok for me?

[hexx · Accepted solution]

I don't believe that it's a good idea to suppress those warnings, but if you really need to do so, you could edit $SPLUNK_HOME/etc/log.cfg and push the threshold of DateParserVerbose to ERROR by adding this line :

category.DateParserVerbose=ERROR

This requires a restart of splunkd to take effect.

Please note that by doing this, you would obfuscate other warnings of this type for other sources, which might be undesirable.

[hexx]

These errors are logged when Splunk is unable to find a time stamp in the event and falls back on something else such as the time stamp of the previous event from that source/host/sourcetype, or the current time.

[Derek]

Yes, I would rather not, but haven't been able to get anywere yet with it.

When does splunkd log the error? If it doesn't parse a timestamp from the raw event and has to use the filename or indexing time?