Splunk Dev

Http Event Collector: Unable to send events through log4j

viranis
Explorer

I have setup the trial version of Splunk enterprise on my machine and have also created a dummy java spring boot service with log4j2 framework. The idea is to capture the logs in splunk from this service using HEC. I did find a nice tutorial that i followed but i still seem to not receive any events in splunk. 

Also, as there are no error messages that i can see, i am not sure what the issue is ? Can someone please guide me to the right place ?

This is the guide that i followed : https://github.com/devadyuti/integration-repo/tree/master/spring-log4j2-splunk

Please let me know if there is anything else i can provide that would be useful. 

pom.xml

 

 

 

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
        <modelVersion>4.0.0</modelVersion>
        <parent>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-parent</artifactId>
                <version>2.4.0</version>
                <relativePath/> <!-- lookup parent from repository -->
        </parent>
        <groupId>com.example</groupId>
        <artifactId>splunk-log4j</artifactId>
        <version>0.0.1-SNAPSHOT</version>
        <name>splunk-log4j</name>
        <description>Demo project for Splunk with springboot</description>

        <properties>
                <java.version>11</java.version>
        </properties>

        <repositories>
                <repository>
                        <id>splunk-artifactory</id>
                        <name>Splunk Releases</name>
                        <!--<url>https://splunk.artifactoryonline.com/artifactory/ext-releases-local</url>-->
                        <url>https://splunk.jfrog.io/splunk/ext-releases-local</url>
                </repository>
        </repositories>

        <dependencies>
                <dependency>
                        <groupId>org.springframework.boot</groupId>
                        <artifactId>spring-boot-starter-web</artifactId>
                        <exclusions>
                                <exclusion>
                                        <groupId>org.springframework.boot</groupId>
                                        <artifactId>spring-boot-starter-logging</artifactId>
                                </exclusion>
                        </exclusions>
                </dependency>

                <dependency>
                        <groupId>org.springframework.boot</groupId>
                        <artifactId>spring-boot-starter-test</artifactId>
                        <scope>test</scope>
                </dependency>

                <dependency>
                        <groupId>org.springframework.boot</groupId>
                        <artifactId>spring-boot-starter-log4j2</artifactId>
                </dependency>

                <dependency>
                        <groupId>org.springframework.boot</groupId>
                        <artifactId>spring-boot-starter-actuator</artifactId>
                </dependency>

                <dependency>
                        <groupId>com.splunk.logging</groupId>
                        <artifactId>splunk-library-javalogging</artifactId>
                        <version>1.8.0</version>
                </dependency>
        </dependencies>

        <build>
                <plugins>
                        <plugin>
                                <groupId>org.springframework.boot</groupId>
                                <artifactId>spring-boot-maven-plugin</artifactId>
                        </plugin>
                </plugins>
        </build>

</project>

 

 

 

 

log4j2.xml

 

 

 

<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
    <Appenders>
        <Console name="console" target="SYSTEM_OUT">
            <PatternLayout
                    pattern="%style{%d{ISO8601}} %highlight{%-5level }[%style{%t}{bright,blue}] %style{%C{10}}{bright,yellow}: %msg%n%throwable" />
        </Console>
        <SplunkHttp
            name="splunkhttp"
            url="http://127.0.0.1:8000/services/collector/event"
            token="xxxxxxxxxxxxxxxxxxxxxx"
            index="http_log_event_collector_idx"
            host="127.0.0.1"
            type="raw"
            sourcetype="_json"
            messageFormat="text"
            disableCertificateValidation="true"
            >
            <PatternLayout pattern="%m" />
        </SplunkHttp>

    </Appenders>

    <Loggers>
        <!-- LOG everything at INFO level -->
        <Root level="trace">
            <AppenderRef ref="console" />
            <AppenderRef ref="splunkhttp" />
        </Root>
    </Loggers>
</Configuration>

 

 

 

 

0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

The HEC port number usually is 8088 rather than 8000.  Correct the url in the log4j.xml and see if you get results.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

viranis
Explorer

I think i have gotten it to work now. One of the issues i see is that in splunk, i had selected the source type as _json. But as soon as i changed it to log4j, i saw the logs. I will play around with this but the 8088 did the trick as well. Not sure how i missed that. Thanks

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The HEC port number usually is 8088 rather than 8000.  Correct the url in the log4j.xml and see if you get results.

---
If this reply helps you, Karma would be appreciated.

viranis
Explorer

Unfortunately, that did not work. I have attached the screenshot from splunk enterprise

 

 

 

splunkHEC.PNG

0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...