Splunk Dev

logging configuration generates index out of bounds error

kidderjc
New Member

I am attempting to integrate a third-party application with an existing log4j implementation into Splunk.  I have what I beleive should be a working appender configuration in my log4j.properties file.  However, when my Tomcat server starts I receive the below index out of bounds error.  I am using logging library version 1.9.0. I'm looking for advice on where to look in order to resolve this.  I have included the appender config for reference.

APPENDER CONFIG:

appender.splunkHEC=com.splunk.logging.HttpEventCollectorLog4jAppender
appender.splunkHEC.name=splunkHEC
appender.splunkHEC.layout=org.apache.log4j.PatternLayout
appender.splunkHEC.layout.ConversionPattern=%d{ISO8601} [%t] %p %c %x - %m%n
appender.splunkHEC.url=<redacted>
appender.splunkHEC.token=<redacted>
appender.splunkHEC.index=ioeng
appender.splunkHEC.source=IIQ_Tomcat
appender.splunkHEC.sourceType=log4j
appender.splunkHEC.batch_size_count=100
appender.splunkHEC.disableCertificateValidation=true


RELEVANT JAVA STACK:

Caused by: java.lang.StringIndexOutOfBoundsException: begin 0, end -1, length 9
at java.base/java.lang.String.checkBoundsBeginEnd(String.java:3319)
at java.base/java.lang.String.substring(String.java:1874)
at org.apache.logging.log4j.util.PropertiesUtil.partitionOnCommonPrefixes(PropertiesUtil.java:555)
at org.apache.logging.log4j.core.config.properties.PropertiesConfigurationBuilder.build(PropertiesConfigurationBuilder.java:156)
at org.apache.logging.log4j.core.config.properties.PropertiesConfigurationFactory.getConfiguration(PropertiesConfigurationFactory.java:56)
at org.apache.logging.log4j.core.config.properties.PropertiesConfigurationFactory.getConfiguration(PropertiesConfigurationFactory.java:35)
at org.apache.logging.log4j.core.config.ConfigurationFactory$Factory.getConfiguration(ConfigurationFactory.java:557)
at org.apache.logging.log4j.core.config.ConfigurationFactory$Factory.getConfiguration(ConfigurationFactory.java:481)
at org.apache.logging.log4j.core.config.ConfigurationFactory.getConfiguration(ConfigurationFactory.java:323)
at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:695)
at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:716)
at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:270)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:155)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:47)
at org.apache.logging.log4j.LogManager.getContext(LogManager.java:196)
at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getContext(AbstractLoggerAdapter.java:137)
at org.apache.logging.log4j.jcl.LogAdapter.getContext(LogAdapter.java:40)
at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:47)
at org.apache.logging.log4j.jcl.LogFactoryImpl.getInstance(LogFactoryImpl.java:40)
at org.apache.logging.log4j.jcl.LogFactoryImpl.getInstance(LogFactoryImpl.java:55)
at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:655)
at sailpoint.web.StartupContextListener.<clinit>(StartupContextListener.java:59)


SERVER DETAILS:

20-Mar-2024 11:52:03.882 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version name: Apache Tomcat/9.0.64
20-Mar-2024 11:52:03.883 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server built: Jun 2 2022 19:08:46 UTC
20-Mar-2024 11:52:03.884 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version number: 9.0.64.0
20-Mar-2024 11:52:03.884 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Name: Linux
20-Mar-2024 11:52:03.885 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Version: 3.10.0-1160.108.1.el7.x86_64
20-Mar-2024 11:52:03.886 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Architecture: amd64
20-Mar-2024 11:52:03.886 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Java Home: /usr/java/jdk-11.0.22
20-Mar-2024 11:52:03.887 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Version: 11.0.22+9-LTS-219
20-Mar-2024 11:52:03.887 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor: Oracle Corporation
Labels (2)
Tags (1)
0 Karma

VatsalJagani
SplunkTrust
SplunkTrust

@kidderjc - I'm no Java expert based on my past experience with log4j to Splunk HEC. If Splunk fails for some reason your solution will encounter a memory issue and may crash.

My Recommendation: Store logs to log files on the server and use Splunk UF to forward the logs to Splunk indexers.

 

I hope this helps!!!

0 Karma
Get Updates on the Splunk Community!

Threat Hunting Unlocked: How to Uplevel Your Threat Hunting With the PEAK Framework ...

WATCH NOWAs AI starts tackling low level alerts, it's more critical than ever to uplevel your threat hunting ...

Splunk APM: New Product Features + Community Office Hours Recap!

Howdy Splunk Community! Over the past few months, we’ve had a lot going on in the world of Splunk Application ...

Index This | Forward, I’m heavy; backward, I’m not. What am I?

April 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...