Character Encoding in log4j

tscanlon — Tue, 22 Oct 2013 23:43:04 GMT

Setting up Splunk I'm getting rsyslog messages showing up fine but when I point a little test log4j app at it I start getting \xabc\x00 type lines showing up. I've tried the usual suspects with varying results. UTF-8 gets the ASCII characters right but still has lots of \xabc\xedf interspersed. UTF-16 is all an asian character set with a couple \x00 (specifially \x00 and no others). LATIN1 gets ascii right but again lots of \xabc\xedf.

edit: says I don't have enough karma to post external links or upload a picture so the image is at
imgur.com/uZaTifZ

Some engineers threw together a quick java program to give me the ability to test log4j.properties and configuring splunk. Below is log4j.properties and the code to make the same program (its setup using gradle).

log4j.properties

  1 log4j.rootLogger=DEBUG, server, FILE
  2 log4j.appender.server=org.apache.log4j.net.SocketAppender
  3 log4j.appender.server.Port=4712
  4 log4j.appender.server.RemoteHost=stg-logs01.sjc01.baynote.net
  5 log4j.appender.server.ReconnectionDelay=10000
  6 log4j.appender.server.layout = org.apache.log4j.PatternLayout
  7 log4j.appender.server.layout.conversionPattern = %m%n
  8 log = /tmp/
  9 log4j.appender.FILE = org.apache.log4j.FileAppender
 10 log4j.appender.FILE.File = ${log}/log.out
 11 log4j.appender.FILE.layout = org.apache.log4j.PatternLayout
 12 log4j.appender.FILE.layout.conversionPattern = %m%n

edit2: I've also tried adding log4j.appender.server.encoding = UTF-8 to the properties file and still getting \xaB\xCD

and the code

  1 package com;
  2
  3 import org.apache.log4j.Logger;
  4
  5 public class Log4JTester {
  6         static Logger logger = Logger.getLogger(Log4JTester.class);
  7
  8         public static void main(String[] args) {
  9                 logger.info("...HI....");
 10         }
 11 }

Re: Character Encoding in log4j

Damien_Dallimor — Wed, 23 Oct 2013 05:05:14 GMT

Firstly ,org.apache.log4j.net.SocketAppender sends serialized LoggingEvent objects over the wire ie: binary data. You need to send textual data to Splunk. So this is not going to work.

Secondly , I would suggest looking at the Splunk log4j appenders on github :

https://github.com/splunk/splunk-library-javalogging

With these appenders you can setup your log4j logging configuration to send your log events to Splunk over raw TCP or via HTTP REST.

Re: Character Encoding in log4j

tscanlon — Wed, 23 Oct 2013 23:33:11 GMT

Thanks looks like I'm going to have to rework the setup.

topic Re: Character Encoding in log4j in Splunk Search

Character Encoding in log4j

Re: Character Encoding in log4j

Re: Character Encoding in log4j