unable to break multi line events in splunk

[luv]

10:32:21,453 INFO [2212] abcdxyz <-|
10:32:21,112 INFO [2212] abcdxyz |
10:32:22,409 INFO [1121] abcdxyz |

10:32:23,003 INFO [1121] abcdxyz |
10:32:23,453 INFO [2212] abcdxyz |
|
............ |
............ |
............ | <---this whole block is taken as the 1st event by splunk
some HTML codes |
............ |
............ |
............ |
<-|
11:09:30,103 INFO [1008] abcdxyz <-|
11:09:30,409 INFO [4560] abcdxyz |
11:10:31,092 INFO [2212] abcdxyz |
11:10:31,470 INFO [2212] abcdxyz | <- this whole block is taken as the 2nd event by splunk

|
............ |
............ |
............ |
Some HTML codes |
............ |
............ |
............ |
<-|
12:09:20,125.....and so on <-|
| <- 4th event
.
.
<-.
. <- 5th event
.
<-.

I don't know why splunk is not breaking the lines when it finds the timestamp instead of that it is combining few lines as if they are single event though they have different timestamps
Is there any way i can split these lines according to the timestamp?

FYI- I am using splunk universal forwarder(Windows) and splunk indexer(Linux)

Thanks 🙂

[lguinn2 · Accepted solution]

I like @mloven_splunk's answer, but I always prefer BREAK_ONLY_BEFORE = <regular expression> to LINE_BREAKER. LINE_BREAKER has potentially nasty side-effects.

[sourcetypenamehere]
BREAK_ONLY_BEFORE =\d{1,2}:\d{2}:\d{2},\d{3}

You may also want to specify the TIME_FORMAT. Here are some links to the documentation:

How timestamp assignment works

Configure timestamp recognition

Finally, if you get the timestamp recognition right, you might not need to specify the linebreaking. Splunk generally breaks on timestamp by default...

[luv]

Thanks Iguinn that worked 🙂

[lguinn2]

Sorry about the typo - I made the correction in my answer. Thanks for the catch.

[mataharry]

yes they were removed by the html formating

"d" shoud be "backslashd"

[lukejadamec]

I'm still learning regex, is the regex you posted missing \ before the d's?

[lukejadamec]

Can you post your configuration for this input from inputs.conf?
See if mloven's line breaker works, and then we'll know if you need help with the timestamp.

[somesoni2]

By default Splunk looks for timestamp with date (your just have time) to automatically split the events based on timestamp. Since, your file shows just the time part, splunk need an extra option to break the events on just time. you would have to add "BREAK_ONLY_BEFORE=\d{2}:\d{2}:\d{2},\d{3}" in props.conf to enable splunk to do eventbreak per your requirement.

[mloven_splunk]

plop this in a props.conf file

[sourcetypenamehere]
LINE_BREAKER = ([\r\n]+)(?\d{1,2}:\d{2}:\d{2},\d{3}

[luv]

It is a java code used to perform banking transactions
Is there anyway i can split it?
this "10:32:21,453" time stamp doesn't count as a valid time by splunk?

[lukejadamec]

This log looks like one of those logs that Splunk needs a little help with.
What program is generating this log?

[luv]

no i don't have props.conf configured for this in the indexer.
splunk is supposed to split it automatically right? as there is no need to tell splunk explicitly where to split the events because many lines have timestamps

[lukejadamec]

How do you have this sourcetype configured in props.conf?