All Apps and Add-ons

Splunk Add-on for JBoss: Why is Splunk combining log entries into one log entry?

Communicator

We are importing logs from a JBoss server (with Splunk Add-on for JBoss installed), and we are noticing that there are several instances where there a few log entries combined into one log entry in Splunk. What would be causing this and how do I fix it? Is this a Splunk issue or an add-on issue?

0 Karma
1 Solution

Explorer

I ran into the same problem, so I figured I'd post my fix here. Skoelpin was right about the line breaking problem. The line breaking and field extraction in the default props.conf doesn't work correctly (at least with my version of jboss). It's expecting a time field without a date, but my jboss logs had both date and time at the beginning of each line.

Here's the format in my logs: 2018-03-29 16:20:31,058

To fix it, put the following in $SPLUNK_HOME/etc/apps//local/props.conf:

[jboss:server:log]
EXTRACT-server = ^\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{3}\s+(?P<log_level>\w+)\s+\[(?P<event_category>[\-\.\w$]+)\]\s(?P<message>[\s\S]+)$
BREAK_ONLY_BEFORE = ^\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{3}

View solution in original post

0 Karma

Explorer

I ran into the same problem, so I figured I'd post my fix here. Skoelpin was right about the line breaking problem. The line breaking and field extraction in the default props.conf doesn't work correctly (at least with my version of jboss). It's expecting a time field without a date, but my jboss logs had both date and time at the beginning of each line.

Here's the format in my logs: 2018-03-29 16:20:31,058

To fix it, put the following in $SPLUNK_HOME/etc/apps//local/props.conf:

[jboss:server:log]
EXTRACT-server = ^\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{3}\s+(?P<log_level>\w+)\s+\[(?P<event_category>[\-\.\w$]+)\]\s(?P<message>[\s\S]+)$
BREAK_ONLY_BEFORE = ^\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{3}

View solution in original post

0 Karma

Explorer

I noticed I accidentally hit the quote button instead of the code button, so the prop.conf lines were incorrect. Fixed to show correctly.

0 Karma

SplunkTrust
SplunkTrust

Your on the right track, but LINE_BREAKER is a better attribute than BREAK_ONLY_BEFORE and you should also set SHOULD_LINEMERGE=false

0 Karma

Explorer

I based the attributes I used on the ones in the default props.conf. I wanted to modify it as little as possible. However, I might look into those attributes in addition.

Here's the default props.conf from the add-on:
[jboss:server:log]
MAXTIMESTAMPLOOKAHEAD = 32

01:59:41,057

EXTRACT-server = ^\d{2}:\d{2}:\d{2},\d{3}\s+(?P\w+)\s+[(?P[-.\w$]+)]\s((?P.+?))\s(?P[\s\S]+)$
BREAKONLYBEFORE = ^\d{2}:\d{2}:\d{2},\d{3}
LOOKUP-severityname = jbossseveritylookup loglevel OUTPUT severity
FIELDALIAS-body = message AS body

FIELDALIAS-subject = event_category AS subject

EVAL-app = "JBoss"

EDIT: The code block is apparently not working for me in comments section...ugg

0 Karma

SplunkTrust
SplunkTrust

It sounds like your events are not correctly line breaking. Can you go to $SPLUNK_HOME/etc/apps/<JBOSS_APP>/local and paste the contents of your props.conf? You should also provide a small set of sample data.

0 Karma

Communicator

Um... I am not seeing a props.conf file.

0 Karma