LINE_BREAKER Help Needed

[babcolee]

I need help breaking the following data into segments. The data is currently lumped together. I have been working with the Splunk Add Data feature to attempt to parse the data correctly

07400 16:31:30.320 Processing 51 log entries in <servername.615494dd0000.dblog> from servername 07784 16:31:30.492 Processing 51 log entries in <servername.615494e00000.dblog> from servername 07400 16:31:30.633 DBLog Summary: time=313ms (total=51, mean time=6.137/rec), Message:(c=32, t=297) Content:(c=5, t=0) NodeStats:(c=1, t=0) VirusScannerStats:(c=13, t=0) 07784 16:31:30.987 DBLog Summary: time=484ms (total=51, mean time=9.490/rec), Message:(c=35, t=469) Content:(c=4, t=0) NodeStats:(c=1, t=0) VirusScannerStats:(c=11, t=0) 07784 16:31:31.213 Processing 51 log entries in <servername.615494e00000.dblog> from servername 07784 16:31:31.278 DBLog Summary: time=62ms (total=51, mean time=1.216/rec), Message:(c=31, t=31) Content:(c=9, t=16) NodeStats:(c=1, t=0) VirusScannerStats:(c=10, t=0) 07784 16:31:31.691 Processing 51 log entries in <servername.615494e20000.dblog> from servername 07400 16:31:31.739 Rule Profiler: writing queued records to the database. 07400 16:31:31.745 Rule Profiler: finished writing queued records to the database. Record count: 53 07784 16:31:31.776 DBLog Summary: time=93ms (total=51, mean time=1.824/rec), Message:(c=31, t=78) Content:(c=6, t=0) NodeStats:(c=2, t=0) VirusScannerStats:(c=12, t=0)

In Regex tester I have used the regex (\d{5}\s+\d{2}:\d{2}:\d{2}.\d{3}\s+Processing 51) to correctly capture where the data needs to be on a new line. I need the event data parsed to look as follows:

07400 16:31:30.320 Processing 51 log entries in <servername.615494dd0000.dblog> from servername
07784 16:31:30.492 Processing 51 log entries in <servername.615494e00000.dblog> from servername 07400 16:31:30.633 DBLog Summary: time=313ms (total=51, mean time=6.137/rec), Message:(c=32, t=297) Content:(c=5, t=0) NodeStats:(c=1, t=0) VirusScannerStats:(c=13, t=0) 07784 16:31:30.987 DBLog Summary: time=484ms (total=51, mean time=9.490/rec), Message:(c=35, t=469) Content:(c=4, t=0) NodeStats:(c=1, t=0) VirusScannerStats:(c=11, t=0)
07784 16:31:31.213 Processing 51 log entries in <servername.615494e00000.dblog> from servername 07784 16:31:31.278 DBLog Summary: time=62ms (total=51, mean time=1.216/rec), Message:(c=31, t=31) Content:(c=9, t=16) NodeStats:(c=1, t=0) VirusScannerStats:(c=10, t=0)
07784 16:31:31.691 Processing 51 log entries in <servername.615494e20000.dblog> from servername 07400 16:31:31.739 Rule Profiler: writing queued records to the database. 07400 16:31:31.745 Rule Profiler: finished writing queued records to the database. Record count: 53 07784 16:31:31.776 DBLog Summary: time=93ms (total=51, mean time=1.824/rec), Message:(c=31, t=78) Content:(c=6, t=0) NodeStats:(c=2, t=0) VirusScannerStats:(c=12, t=0)

I have tried LINE_BREAKER=([\r\n]+), BREAK_ONLY_BEFORE, MUST_BREAK_AFTER, MUST_NOT_BREAK_BEFORE along with using the regex shown above in the Splunk Wizard will not break the data as needed.

Thanks

[richgalloway · Accepted solution]

A couple of key things to remember about LINE_BREAKER are: 1) it must contain a capture group (which can be empty); and 2) the text that matches the capture group is always discarded.

Have you tried this?

LINE_BREAKER = ()\d{5}\s

[babcolee]

I was not aware of the empty capturing group. Thank you, your solution worked!