Splunk Search

Character Encoding in log4j

tscanlon
Engager

Setting up Splunk I'm getting rsyslog messages showing up fine but when I point a little test log4j app at it I start getting \xabc\x00 type lines showing up. I've tried the usual suspects with varying results. UTF-8 gets the ASCII characters right but still has lots of \xabc\xedf interspersed. UTF-16 is all an asian character set with a couple \x00 (specifially \x00 and no others). LATIN1 gets ascii right but again lots of \xabc\xedf.

edit: says I don't have enough karma to post external links or upload a picture so the image is at
imgur.com/uZaTifZ

Some engineers threw together a quick java program to give me the ability to test log4j.properties and configuring splunk. Below is log4j.properties and the code to make the same program (its setup using gradle).

log4j.properties

  1 log4j.rootLogger=DEBUG, server, FILE
  2 log4j.appender.server=org.apache.log4j.net.SocketAppender
  3 log4j.appender.server.Port=4712
  4 log4j.appender.server.RemoteHost=stg-logs01.sjc01.baynote.net
  5 log4j.appender.server.ReconnectionDelay=10000
  6 log4j.appender.server.layout = org.apache.log4j.PatternLayout
  7 log4j.appender.server.layout.conversionPattern = %m%n
  8 log = /tmp/
  9 log4j.appender.FILE = org.apache.log4j.FileAppender
 10 log4j.appender.FILE.File = ${log}/log.out
 11 log4j.appender.FILE.layout = org.apache.log4j.PatternLayout
 12 log4j.appender.FILE.layout.conversionPattern = %m%n

edit2: I've also tried adding log4j.appender.server.encoding = UTF-8 to the properties file and still getting \xaB\xCD

and the code

  1 package com;
  2
  3 import org.apache.log4j.Logger;
  4
  5 public class Log4JTester {
  6         static Logger logger = Logger.getLogger(Log4JTester.class);
  7
  8         public static void main(String[] args) {
  9                 logger.info("...HI....");
 10         }
 11 }
Tags (2)
0 Karma
1 Solution

Damien_Dallimor
Ultra Champion

Firstly ,org.apache.log4j.net.SocketAppender sends serialized LoggingEvent objects over the wire ie: binary data. You need to send textual data to Splunk. So this is not going to work.

Secondly , I would suggest looking at the Splunk log4j appenders on github :

https://github.com/splunk/splunk-library-javalogging

With these appenders you can setup your log4j logging configuration to send your log events to Splunk over raw TCP or via HTTP REST.

View solution in original post

Damien_Dallimor
Ultra Champion

Firstly ,org.apache.log4j.net.SocketAppender sends serialized LoggingEvent objects over the wire ie: binary data. You need to send textual data to Splunk. So this is not going to work.

Secondly , I would suggest looking at the Splunk log4j appenders on github :

https://github.com/splunk/splunk-library-javalogging

With these appenders you can setup your log4j logging configuration to send your log events to Splunk over raw TCP or via HTTP REST.

tscanlon
Engager

Thanks looks like I'm going to have to rework the setup.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...