topic Splunk splits multi-line log4j xml event in Dashboards & Visualizations

Splunk splits multi-line log4j xml event

NK_1 — Sat, 19 Jul 2014 01:08:49 GMT

I get 2 events (one for the "event" part and 1 for the "message" part) instead of 1 event.

Raw logged lines:



<log4j:event category="com.myapp.shr.util.sql.SelectStatement" timestamp="1405542493235" level="DEBUG" thread="xxx">

<log4j:message xxxxCategory="1" messageKey="LOG_SELECTSTATEMENTEXECUTEQUERYRETURNED" arg1="jdbc:oracle:thin:@xxx:1521:yyy" arg2="Oracle" arg3="  0ms" arg4="SELECT  A,B,C FROM TABLE1" arg5="xxx" bracketingId=" "><![CDATA[Duration:   0ms  SQL: SELECT  A,B,C FROM TABLE1

 Server: jdbc:oracle:thin:@xxx:1521:yyy:

 DB: Oracle

 xxx] ]]></log4j:message>

</log4j:event>

My C:\Program Files\Splunk\etc\system\local\props.conf contains:

[source::...\\logs\\YYY.log4j]
sourcetype = YYY
TIME_PREFIX = timestamp=
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)<log4j:event

YYY.log4j logfiles are getting sent via Splunk Universal Forwarder (tcp).

Shouldn't the above settings combine the lines to create 1 event?

Re: Splunk splits multi-line log4j xml event

martin_mueller — Sat, 19 Jul 2014 09:58:35 GMT

Try splitting up your props.conf like this:

[source::...\\logs\\YYY.log4j]
sourcetype = YYY

[YYY]
TIME_PREFIX = timestamp=
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)<log4j:event

Re: Splunk splits multi-line log4j xml event

NK_1 — Sat, 19 Jul 2014 19:05:58 GMT

Adding the [YYY] stanza still makes Splunk use the old/default YYY split events. I tried using a new sourcetype name instead of YYY, and it does not get used. Should I be using a different props.conf file, or should I try to make Splunk "forget" that it ever saw a wrong YYY format?

Re: Splunk splits multi-line log4j xml event

NK_1 — Sun, 20 Jul 2014 23:05:17 GMT

Splunk had "learned" and cached some bad formats for sourcetype YYY in props.conf and sourcetypes.conf under etc\apps\learned\local\

Clearing those out made the YYY stanza in props.conf under etc\system\local\ work properly.

Thanks!