Dashboards & Visualizations
Highlighted

Splunk splits multi-line log4j xml event

Path Finder

I get 2 events (one for the "event" part and 1 for the "message" part) instead of 1 event.

Raw logged lines:


<log4j:event category="com.myapp.shr.util.sql.SelectStatement" timestamp="1405542493235" level="DEBUG" thread="xxx">
<log4j:message xxxxCategory="1" messageKey="LOG_SELECTSTATEMENTEXECUTEQUERYRETURNED" arg1="jdbc:oracle:thin:@xxx:1521:yyy" arg2="Oracle" arg3=" 0ms" arg4="SELECT A,B,C FROM TABLE1" arg5="xxx" bracketingId=" "><![CDATA[Duration: 0ms SQL: SELECT A,B,C FROM TABLE1
Server: jdbc:oracle:thin:@xxx:1521:yyy:
DB: Oracle
xxx] ]]></log4j:message>
</log4j:event>

My C:\Program Files\Splunk\etc\system\local\props.conf contains:

[source::...\\logs\\YYY.log4j]
sourcetype = YYY
TIME_PREFIX = timestamp=
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)<log4j:event

YYY.log4j logfiles are getting sent via Splunk Universal Forwarder (tcp).

Shouldn't the above settings combine the lines to create 1 event?

Highlighted

Re: Splunk splits multi-line log4j xml event

SplunkTrust
SplunkTrust

Try splitting up your props.conf like this:

[source::...\\logs\\YYY.log4j]
sourcetype = YYY

[YYY]
TIME_PREFIX = timestamp=
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)<log4j:event

View solution in original post

Highlighted

Re: Splunk splits multi-line log4j xml event

Path Finder

Adding the [YYY] stanza still makes Splunk use the old/default YYY split events. I tried using a new sourcetype name instead of YYY, and it does not get used. Should I be using a different props.conf file, or should I try to make Splunk "forget" that it ever saw a wrong YYY format?

0 Karma
Highlighted

Re: Splunk splits multi-line log4j xml event

Path Finder

Splunk had "learned" and cached some bad formats for sourcetype YYY in props.conf and sourcetypes.conf under etc\apps\learned\local\

Clearing those out made the YYY stanza in props.conf under etc\system\local\ work properly.

Thanks!