Getting Data In

log4j socket appender and Splunk?

ljoshi
Splunk Employee
Splunk Employee

Does Splunk work with a log4j socket appender? ( not the rolling file one). How?

Tags (1)

nappana
New Member

i have .log file generate by using log4j. now my task is to pass this .log file into splunk dashboard. can anyone explain me step by step how to pass .log file into splunk dashboard.

,

0 Karma

Damien_Dallimor
Ultra Champion

Check out SplunkJavaLogging

Log4j and Logback appenders to send events to Splunk via HTTP REST or Raw TCP
Helper classes for formatting log events in a best practice semantic format for Splunk

0 Karma

unitedmarsupial
Path Finder

Unfortunately, that library only works with log4j-2.x. Some of us are -- for the sins we've committed in prior lives -- still stuck with log4j-1.x

0 Karma

LauraBre
Communicator

I read your subject and I hava a question because I hava a similar problem. When I read my events in Splunk the data format is different. There are a lot of characters which haven't be it. Can you show me how you are logging your events. Me, I do it but I think it is bad:

LOGGER_SPLUNK.info("Requested serv. : D2T, Nb PAN : " + nbPan +", Requester : " + body.get(NlvValue.REQUESTER_ID) +", User : " + body.get(NlvValue.REQUESTER_ID)+"_ow, Host : "+host+" ServiceName : ");//+ exchange.getProperty(ConstantUtil.SERVICE_NAME));

this is my log4j.properties:

log4j.logger.net.awl.bfi.TokenizerWatchdogSplunk=info,watchdogSplunkSocket
log4j.appender.watchdogSplunkSocket=org.apache.log4j.net.SocketAppender
log4j.appender.watchdogSplunkSocket.remoteHost=odpcil01b
log4j.appender.watchdogSplunkSocket.port=5541
log4j.appender.watchdogSplunkSocket.locationInfo=false
log4j.appender.watchdogSplunkSocket.layout=org.apache.log4j.PatternLayout 
log4j.appender.watchdogSplunkSocket.layout.ConversionPattern = [%-5p][%d{dd/MM/yyyy HH:mm:ss}][%c][%F]%m%n

Thanks by advance,

Laura

0 Karma

mawalters1
Engager

Fully agree with bchen, you will get serialized garble in your messages on the splunk index using SocketAppender. Log4j properties example below just alter SyslogHost values. Port is optional, but useful to create various index sources.

We use another appender to created log files of same data, used to fill long term analysis, feed those to splunk in different index.

example for syslog appender log4j log4j.appender.SPLUNKiT=org.apache.log4j.net.SyslogAppender log4j.appender.SPLUNKiT.SyslogHost=[:CustomPort] log4j.appender.SPLUNKiT.layout=org.apache.log4j.PatternLayout log4j.appender.SPLUNKiT.layout.ConversionPattern=sv-cdr-posted - %m log4j.appender.SPLUNKiT.Facility=USER

bchen
Splunk Employee
Splunk Employee

It's unlikely you want to use SocketAppender with Splunk, since it sends a serialized Java object, LoggingEvent, which is meant for something like SocketNode to receive and deserialize.

Something that may have more sensible data is to use SyslogAppender. (though I haven't tried it personally)

0 Karma

dwaddle
SplunkTrust
SplunkTrust

Splunk can listen on a TCP socket for an arbitrary stream of bytes. This should include the output of log4j's socket appender. The inputs.conf stanza for this is similar to:

[tcp://7070]
connection_host=dns
sourcetype=log4j

There's at least one caveat with this approach - your log4j data is not persisted anywhere until it is indexed. If the connectivity between your log4j app and splunk is disrupted, or if the Splunk indexer is down for some reason -- you could lose event data. A forwarder on the log4j host, reading a file produced by the log4j app is more robust in this example.

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...