topic Re: Splunk Add-on for JBoss: Why is Splunk combining log entries into one log entry? in All Apps and Add-ons

Splunk Add-on for JBoss: Why is Splunk combining log entries into one log entry?

cboillot — Tue, 11 Jul 2017 15:56:05 GMT

We are importing logs from a JBoss server (with Splunk Add-on for JBoss installed), and we are noticing that there are several instances where there a few log entries combined into one log entry in Splunk. What would be causing this and how do I fix it? Is this a Splunk issue or an add-on issue?

Re: Splunk Add-on for JBoss: Why is Splunk combining log entries into one log entry?

skoelpin — Tue, 11 Jul 2017 15:58:51 GMT

It sounds like your events are not correctly line breaking. Can you go to $SPLUNK_HOME/etc/apps/<JBOSS_APP>/local and paste the contents of your props.conf? You should also provide a small set of sample data.

Re: Splunk Add-on for JBoss: Why is Splunk combining log entries into one log entry?

cboillot — Tue, 11 Jul 2017 16:53:06 GMT

Um... I am not seeing a props.conf file.

Re: Splunk Add-on for JBoss: Why is Splunk combining log entries into one log entry?

rmills1 — Thu, 29 Mar 2018 20:23:00 GMT

I ran into the same problem, so I figured I'd post my fix here. Skoelpin was right about the line breaking problem. The line breaking and field extraction in the default props.conf doesn't work correctly (at least with my version of jboss). It's expecting a time field without a date, but my jboss logs had both date and time at the beginning of each line.

Here's the format in my logs: 2018-03-29 16:20:31,058

To fix it, put the following in $SPLUNK_HOME/etc/apps//local/props.conf:

[jboss:server:log]
EXTRACT-server = ^\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{3}\s+(?P<log_level>\w+)\s+\[(?P<event_category>[\-\.\w$]+)\]\s(?P<message>[\s\S]+)$
BREAK_ONLY_BEFORE = ^\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{3}

Re: Splunk Add-on for JBoss: Why is Splunk combining log entries into one log entry?

rmills1 — Mon, 02 Apr 2018 18:52:50 GMT

I noticed I accidentally hit the quote button instead of the code button, so the prop.conf lines were incorrect. Fixed to show correctly.

Re: Splunk Add-on for JBoss: Why is Splunk combining log entries into one log entry?

skoelpin — Mon, 02 Apr 2018 18:56:57 GMT

Your on the right track, but LINE_BREAKER is a better attribute than BREAK_ONLY_BEFORE and you should also set SHOULD_LINEMERGE=false

Re: Splunk Add-on for JBoss: Why is Splunk combining log entries into one log entry?

rmills1 — Tue, 29 Sep 2020 18:50:49 GMT

I based the attributes I used on the ones in the default props.conf. I wanted to modify it as little as possible. However, I might look into those attributes in addition.

Here's the default props.conf from the add-on:
[jboss:server:log]
MAX_TIMESTAMP_LOOKAHEAD = 32

01:59:41,057

EXTRACT-server = ^\d{2}:\d{2}:\d{2},\d{3}\s+(?P\w+)\s+[(?P[-.\w$]+)]\s((?P.+?))\s(?P[\s\S]+)$
BREAK_ONLY_BEFORE = ^\d{2}:\d{2}:\d{2},\d{3}
LOOKUP-severity_name = jboss_severity_lookup log_level OUTPUT severity
FIELDALIAS-body = message AS body

FIELDALIAS-subject = event_category AS subject

EVAL-app = "JBoss"

EDIT: The code block is apparently not working for me in comments section...ugg