Archive

Character Encoding in log4j

Engager

Setting up Splunk I'm getting rsyslog messages showing up fine but when I point a little test log4j app at it I start getting \xabc\x00 type lines showing up. I've tried the usual suspects with varying results. UTF-8 gets the ASCII characters right but still has lots of \xabc\xedf interspersed. UTF-16 is all an asian character set with a couple \x00 (specifially \x00 and no others). LATIN1 gets ascii right but again lots of \xabc\xedf.

edit: says I don't have enough karma to post external links or upload a picture so the image is at
imgur.com/uZaTifZ

Some engineers threw together a quick java program to give me the ability to test log4j.properties and configuring splunk. Below is log4j.properties and the code to make the same program (its setup using gradle).

log4j.properties

  1 log4j.rootLogger=DEBUG, server, FILE
  2 log4j.appender.server=org.apache.log4j.net.SocketAppender
  3 log4j.appender.server.Port=4712
  4 log4j.appender.server.RemoteHost=stg-logs01.sjc01.baynote.net
  5 log4j.appender.server.ReconnectionDelay=10000
  6 log4j.appender.server.layout = org.apache.log4j.PatternLayout
  7 log4j.appender.server.layout.conversionPattern = %m%n
  8 log = /tmp/
  9 log4j.appender.FILE = org.apache.log4j.FileAppender
 10 log4j.appender.FILE.File = ${log}/log.out
 11 log4j.appender.FILE.layout = org.apache.log4j.PatternLayout
 12 log4j.appender.FILE.layout.conversionPattern = %m%n

edit2: I've also tried adding log4j.appender.server.encoding = UTF-8 to the properties file and still getting \xaB\xCD

and the code

  1 package com;
  2
  3 import org.apache.log4j.Logger;
  4
  5 public class Log4JTester {
  6         static Logger logger = Logger.getLogger(Log4JTester.class);
  7
  8         public static void main(String[] args) {
  9                 logger.info("...HI....");
 10         }
 11 }
Tags (2)
0 Karma
1 Solution

Ultra Champion

Firstly ,org.apache.log4j.net.SocketAppender sends serialized LoggingEvent objects over the wire ie: binary data. You need to send textual data to Splunk. So this is not going to work.

Secondly , I would suggest looking at the Splunk log4j appenders on github :

https://github.com/splunk/splunk-library-javalogging

With these appenders you can setup your log4j logging configuration to send your log events to Splunk over raw TCP or via HTTP REST.

View solution in original post

Ultra Champion

Firstly ,org.apache.log4j.net.SocketAppender sends serialized LoggingEvent objects over the wire ie: binary data. You need to send textual data to Splunk. So this is not going to work.

Secondly , I would suggest looking at the Splunk log4j appenders on github :

https://github.com/splunk/splunk-library-javalogging

With these appenders you can setup your log4j logging configuration to send your log events to Splunk over raw TCP or via HTTP REST.

View solution in original post

Engager

Thanks looks like I'm going to have to rework the setup.

0 Karma