What log4j ConversionPattern should I use to match splunk log4j sourcetype?

[twhitbeck]

I have a webapp running in Tomcat. I'm using Log4j for my logging, and whenever I load the .log file in Splunk it is not able to determine it's sourcetype. It properly extracts the timestamp, but the rest of the log event is lumped together under the field "event". I want to extract the fields that are in my log (thread, priority, message).

According to Splunk documentation there is a log4j pretrained sourcetype. The log should be of the format: 2005-03-07 16:44:03,110 53223013 [PoolThread-0] INFO [STDOUT] got some property... Well I'm using a log4j ConversionPattern of %d %r [%t] %-5p [STDOUT] %m%n and it's producing log output like: 2012-12-21 09:51:00,937 81 [localhost-startStop-1] INFO [STDOUT] Quartz Scheduler v.1.7.3 created. which looks very similar to the log4j layout prescribed in the Splunk docs, but Splunk is still not recognizing the sourcetype and extracting the fields.

I know I can define my own sourcetype and configure the fields to be extracted with regular expressions, but I didn't want to go through all the hassle if Splunk has already done the work for me.

What log4j conversionpattern should I be using so that Splunk will recognize it as a log4j sourcetype and extract the proper fields?

[emiller42]

If you've got control over your log4j configuration, you can bypass the field extraction entirely by using explicit key/value pairs.

%d{ISO8601} logLevel=%p thread=\"%t\" category=%c %m%n

would output:

2012-12-21 09:51:00,937 logLevel=INFO thread="localhost-startStop-1" category=STDOUT Quartz Scheduler v.1.7.3 created.

Splunk will automatically detect the key/value pairs in the above. If your app uses the MDC with log4J, you can easily add more context this way.

[thefosk]

+1, I'm facing the same issue