We want to use splunk-library-javalogging to send logs via Log4j to Splunk Service
Environment:
Spark with log4j2 in Azure Databricks ----> Splunk Enterprise
The config file log4j2.xml
<?xml version="1.0" encoding="UTF-8"?><Configuration status="INFO" packages="com.splunk.logging,com.databricks.logging.log4j" shutdownHook="disable">
<Appenders>
...
<SplunkHttp name="http-input"
url="https://url-service"
token="xxxx-xxxx-xxxx-xxx-xxx--xxxx"
host=""
index="my-index"
source="spark-work"
sourcetype="httpevent"
messageFormat="text"
middleware="HttpEventCollectorUnitTestMiddleware"
connect_timeout="5000"
termination_timeout="1000"
disableCertificateValidation="true">
<PatternLayout pattern="%m%n"/>
</SplunkHttp>
</Appenders>
<Loggers>
<Root level="INFO">
...
</Root>
...
<Logger name="splunk.log4j" level="DEBUG">
<AppenderRef ref="http-input"/>
</Logger>
</Loggers>
</Configuration>
We use the library splunk-library-javalogging:
splunk-library-javalogging-1.11.8.jar with
Currently we based the configuration from this example:
https://github.com/splunk/splunk-library-javalogging/blob/main/src/test/resources/log4j2.xml
Currently it doesn't work. We checked HEC via curl send a message from Databricks to Splunk HEC and receive without problem.
Does anyone have any experience or can help us with some guidance or advice?
Thanks
Hi @davidco
It'd be worth validating the Splunk receiving end and the logs available. Please could you check for HEC errors using:
index=_internal reply!=0 HttpInputDataHandler
For more info on reply codes see https://docs.splunk.com/Documentation/Splunk/9.4.1/Data/TroubleshootHTTPEventCollector
Any error reply codes here may provide more insights.
🌟 Did this answer help you? If so, please consider:
Your feedback encourages the volunteers in this community to continue contributing
Hello @livehybrid ,
First, thanks for your help.
I tried the query, but it didn't work. I mean, I got no information.
I even tested the HEC via curl, and everything seems normal.
@davidco- Did you check connectivity from Spark server to Splunk service on splunk HEC port?
* via telnet or curl
Hello @VatsalJagani ,
Yes, we checked via Curl
curl -k -X POST 'https://hec-splunk.xxxxx.net/services/collector/event' --header 'Authorization: Splunk xxxx-xxxx-xxxx-xxx-xxxx' -d '{"sourcetype": "my_sample_data", "event": "2025-04-23-Test"}'
Result:
{"text":"Success","code":0}%
And we can see the event in Splunk
Then I think definitely it something related to Log4j configuration or on Spark/Java side in which I have 0 experience, so I'm sorry I won't be able to help you, but I hope someone else in the community will be able to help.