Getting Data In

Can I view related parts of a multiline log entry?

ripper234
Explorer

Some of the long entries my app makes are composed of multiple lines.
I would like to keep it this way (a log line can be 10+ lines sometimes, has lots of info that can't be condensed into a single line).

Is there a way, given a specific text match, to view text lines near it?

E.g. supposed that my log format looks like this:

2013-01-07 13:28:27,325 INFO  (LoggerName) Something is wrong with Website http://foo.com/
Now follow a few important details.
A few more details.
2013-01-07 13:28:27,325 INFO  (LoggerName) Another log entry

If I search in splunk for "foo.com", and only find the first line without the details - how can I view the rest of the details?

I prefer a solution that won't force me to change the format of my log messages.

Tags (1)
0 Karma
1 Solution

emiller42
Motivator

Are the multi-line events being broken into multiple events in Splunk?

Event 1:
2013-01-07 13:28:27,325 INFO  (LoggerName) Something is wrong with Website http://foo.com/
Event 2:
Now follow a few important details.
Event 3:
A few more details.
Event 4:
2013-01-07 13:28:27,325 INFO  (LoggerName) Another log entry

If so, you can fix that where it keeps the whole thing as one event.

In your props.conf, you want to add the following config to the appropriate stanza for this sourcetype:

LINE_BREAKER = ([\r\n]+)\d+-\d+-\d+\s\d+:\d+:\d+,\d+ 
SHOULD_LINEMERGE = false

This tells it not to automatically break events, and instead only break when it encounters a new line starting with a timestamp. (What the regex matches)

Then when new log lines are indexed, multi-line events will be kept as a single event, keeping the context you want:

Event 1:
2013-01-07 13:28:27,325 INFO  (LoggerName) Something is wrong with Website http://foo.com/
Now follow a few important details.
A few more details.
Event 2:
2013-01-07 13:28:27,325 INFO  (LoggerName) Another log entry

View solution in original post

emiller42
Motivator

Are the multi-line events being broken into multiple events in Splunk?

Event 1:
2013-01-07 13:28:27,325 INFO  (LoggerName) Something is wrong with Website http://foo.com/
Event 2:
Now follow a few important details.
Event 3:
A few more details.
Event 4:
2013-01-07 13:28:27,325 INFO  (LoggerName) Another log entry

If so, you can fix that where it keeps the whole thing as one event.

In your props.conf, you want to add the following config to the appropriate stanza for this sourcetype:

LINE_BREAKER = ([\r\n]+)\d+-\d+-\d+\s\d+:\d+:\d+,\d+ 
SHOULD_LINEMERGE = false

This tells it not to automatically break events, and instead only break when it encounters a new line starting with a timestamp. (What the regex matches)

Then when new log lines are indexed, multi-line events will be kept as a single event, keeping the context you want:

Event 1:
2013-01-07 13:28:27,325 INFO  (LoggerName) Something is wrong with Website http://foo.com/
Now follow a few important details.
A few more details.
Event 2:
2013-01-07 13:28:27,325 INFO  (LoggerName) Another log entry

ripper234
Explorer

Interesting direction, I think it will do the trick. I think the lines are all separate events.

Will try it out when I get the chance - thanks for the quick answer!

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Can’t Make It to Boston? Stream .conf25 and Learn with Haya Husain

Boston may be buzzing this September with Splunk University and .conf25, but you don’t have to pack a bag to ...

Splunk Lantern’s Guide to The Most Popular .conf25 Sessions

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Unlock What’s Next: The Splunk Cloud Platform at .conf25

In just a few days, Boston will be buzzing as the Splunk team and thousands of community members come together ...