Getting Data In
Highlighted

unable to break multi line events in splunk

Explorer

10:32:21,453 INFO [2212] abcdxyz <-|
10:32:21,112 INFO [2212] abcdxyz |
10:32:22,409 INFO [1121] abcdxyz |

10:32:23,003 INFO [1121] abcdxyz |
10:32:23,453 INFO [2212] abcdxyz |
|
............ |
............ |
............ | <---this whole block is taken as the 1st event by splunk
some HTML codes |
............ |
............ |
............ |
<-|
11:09:30,103 INFO [1008] abcdxyz <-|
11:09:30,409 INFO [4560] abcdxyz |
11:10:31,092 INFO [2212] abcdxyz |
11:10:31,470 INFO [2212] abcdxyz | <- this whole block is taken as the 2nd event by splunk

|
............ |
............ |
............ |
Some HTML codes |
............ |
............ |
............ |
<-|
12:09:20,125.....and so on <-|
| <- 4th event
.
.
<-.
. <- 5th event
.
<-.

I don't know why splunk is not breaking the lines when it finds the timestamp instead of that it is combining few lines as if they are single event though they have different timestamps
Is there any way i can split these lines according to the timestamp?

FYI- I am using splunk universal forwarder(Windows) and splunk indexer(Linux)

Thanks 🙂

0 Karma
Highlighted

Re: unable to break multi line events in splunk

Super Champion

How do you have this sourcetype configured in props.conf?

0 Karma
Highlighted

Re: unable to break multi line events in splunk

Explorer

no i don't have props.conf configured for this in the indexer.
splunk is supposed to split it automatically right? as there is no need to tell splunk explicitly where to split the events because many lines have timestamps

0 Karma
Highlighted

Re: unable to break multi line events in splunk

Super Champion

This log looks like one of those logs that Splunk needs a little help with.
What program is generating this log?

0 Karma
Highlighted

Re: unable to break multi line events in splunk

Explorer

It is a java code used to perform banking transactions
Is there anyway i can split it?
this "10:32:21,453" time stamp doesn't count as a valid time by splunk?

0 Karma
Highlighted

Re: unable to break multi line events in splunk

Splunk Employee
Splunk Employee

plop this in a props.conf file

[sourcetypenamehere]
LINE_BREAKER = ([\r\n]+)(?\d{1,2}:\d{2}:\d{2},\d{3}

Highlighted

Re: unable to break multi line events in splunk

SplunkTrust
SplunkTrust

By default Splunk looks for timestamp with date (your just have time) to automatically split the events based on timestamp. Since, your file shows just the time part, splunk need an extra option to break the events on just time. you would have to add "BREAKONLYBEFORE=\d{2}:\d{2}:\d{2},\d{3}" in props.conf to enable splunk to do eventbreak per your requirement.

0 Karma
Highlighted

Re: unable to break multi line events in splunk

Super Champion

Can you post your configuration for this input from inputs.conf?
See if mloven's line breaker works, and then we'll know if you need help with the timestamp.

0 Karma
Highlighted

Re: unable to break multi line events in splunk

Legend

I like @mlovensplunk's answer, but I always prefer `BREAKONLYBEFORE = toLINEBREAKER.LINE_BREAKER` has potentially nasty side-effects.

[sourcetypenamehere]
BREAK_ONLY_BEFORE =\d{1,2}:\d{2}:\d{2},\d{3}

You may also want to specify the TIME_FORMAT. Here are some links to the documentation:

How timestamp assignment works

Configure timestamp recognition

Finally, if you get the timestamp recognition right, you might not need to specify the linebreaking. Splunk generally breaks on timestamp by default...

View solution in original post

0 Karma
Highlighted

Re: unable to break multi line events in splunk

Super Champion

I'm still learning regex, is the regex you posted missing \ before the d's?

0 Karma