Getting Data In

Can I view related parts of a multiline log entry?

ripper234
Explorer

Some of the long entries my app makes are composed of multiple lines.
I would like to keep it this way (a log line can be 10+ lines sometimes, has lots of info that can't be condensed into a single line).

Is there a way, given a specific text match, to view text lines near it?

E.g. supposed that my log format looks like this:

2013-01-07 13:28:27,325 INFO  (LoggerName) Something is wrong with Website http://foo.com/
Now follow a few important details.
A few more details.
2013-01-07 13:28:27,325 INFO  (LoggerName) Another log entry

If I search in splunk for "foo.com", and only find the first line without the details - how can I view the rest of the details?

I prefer a solution that won't force me to change the format of my log messages.

Tags (1)
0 Karma
1 Solution

emiller42
Motivator

Are the multi-line events being broken into multiple events in Splunk?

Event 1:
2013-01-07 13:28:27,325 INFO  (LoggerName) Something is wrong with Website http://foo.com/
Event 2:
Now follow a few important details.
Event 3:
A few more details.
Event 4:
2013-01-07 13:28:27,325 INFO  (LoggerName) Another log entry

If so, you can fix that where it keeps the whole thing as one event.

In your props.conf, you want to add the following config to the appropriate stanza for this sourcetype:

LINE_BREAKER = ([\r\n]+)\d+-\d+-\d+\s\d+:\d+:\d+,\d+ 
SHOULD_LINEMERGE = false

This tells it not to automatically break events, and instead only break when it encounters a new line starting with a timestamp. (What the regex matches)

Then when new log lines are indexed, multi-line events will be kept as a single event, keeping the context you want:

Event 1:
2013-01-07 13:28:27,325 INFO  (LoggerName) Something is wrong with Website http://foo.com/
Now follow a few important details.
A few more details.
Event 2:
2013-01-07 13:28:27,325 INFO  (LoggerName) Another log entry

View solution in original post

emiller42
Motivator

Are the multi-line events being broken into multiple events in Splunk?

Event 1:
2013-01-07 13:28:27,325 INFO  (LoggerName) Something is wrong with Website http://foo.com/
Event 2:
Now follow a few important details.
Event 3:
A few more details.
Event 4:
2013-01-07 13:28:27,325 INFO  (LoggerName) Another log entry

If so, you can fix that where it keeps the whole thing as one event.

In your props.conf, you want to add the following config to the appropriate stanza for this sourcetype:

LINE_BREAKER = ([\r\n]+)\d+-\d+-\d+\s\d+:\d+:\d+,\d+ 
SHOULD_LINEMERGE = false

This tells it not to automatically break events, and instead only break when it encounters a new line starting with a timestamp. (What the regex matches)

Then when new log lines are indexed, multi-line events will be kept as a single event, keeping the context you want:

Event 1:
2013-01-07 13:28:27,325 INFO  (LoggerName) Something is wrong with Website http://foo.com/
Now follow a few important details.
A few more details.
Event 2:
2013-01-07 13:28:27,325 INFO  (LoggerName) Another log entry

ripper234
Explorer

Interesting direction, I think it will do the trick. I think the lines are all separate events.

Will try it out when I get the chance - thanks for the quick answer!

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Calling All Security Pros: Ready to Race Through Boston?

Hey Splunkers, .conf25 is heading to Boston and we’re kicking things off with something bold, competitive, and ...

Beyond Detection: How Splunk and Cisco Integrated Security Platforms Transform ...

Financial services organizations face an impossible equation: maintain 99.9% uptime for mission-critical ...

Customer success is front and center at .conf25

Hi Splunkers, If you are not able to be at .conf25 in person, you can still learn about all the latest news ...