How to configure Splunk to ignore a timestamp to count two lines of data as one event?

[MikeBertelsen]

Here is a sampling of the log data. Lines 1-3 had no issues, but line 4 issues a warning and dumps the problem onto line 5. Splunk needs to treat line 5 as part of Line 4 and not index the timestamp for line 5. Suggestions please.

2015-09-11 04:24:05,568 INFO  [nexj.model.class.LDAPGRP] (WorkManager(2)-91:dispatcher ) End getSnapshot for LDAPGRP
2015-09-11 04:24:05,568 INFO  [nexj.model.class.LDAPUSR] (WorkManager(2)-91:dispatcher ) Start getSnapshot for LDAPUSR.
2015-09-11 04:24:05,977 INFO  [nexj.model.class.LDAPUSR] (WorkManager(2)-91:dispatcher ) End getSnapshot for LDAPUSR
2015-09-11 04:42:47,652 WARN  [nexj.core.rpc.http.session.SessionManager] (servername%2F-IPAdress:/project-8009-7:/apex) Acquring lock exceeded max wait time of 30720s. Diagnostic dump:
2015-09-10 11:39:47.312: (Thread: servername%2F-IPAdress:/project-8009-15 Owner: Thread[servername%2F-IPAdress:/project-8009-15,5,jboss] Lock Count: 1 Ref Count: 0 Session: 488E74B020DAC9F33CD503F058947C6A.server) Acqured lock

[somesoni2 · Accepted solution]

Give this a try in your sourcetype definition in props.conf (on Indexer/Heavy Forwarder)

[YourSourceType]
SHOULD_LINEMERGE=false
NO_BINARY_CHECK=true
LINE_BREAKER=([\r\n]+)(?=\d+-\d+-\d+ \d+:\d+:\d+,\d+ \w+)
TIME_FORMAT=%Y-%m-%d %H:%M:%S,%Q
MAX_TIMESTAMP_LOOKAHEAD=30

[MikeBertelsen]

This did work. Can i get a breakdown of the various lines and what they do?
I get Time_format but the others are unclear.

Thanks,
Mike

[somesoni2]

LINE_BREAKER - provides a delimiter which will break input stream into separate events, default is new line.
I've set it to match new line char + string with this format 2015-09-11 04:24:05,568 INFO. Thus excluding intermediate timestamp which doesn't have log level.
SHOULD_LINEMERGE=provide where multiple lines should be merged into one events. Since I'm using custom LINE_BREAKER, it should be false
MAX_TIMESTAMP_LOOKAHEAD=After Timestamp is location, for how many characters the timestamp value should be looked upon. Based on your date (23 char), I just set to near by whole figure of 30.

[edrivera3]

Use a regex in props.conf.

[edrivera3]

Hope this helps:
2015-09-11 04:24:05,977 INFO : \d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{3}\sINFO|WARN