Http Event Collector: Unable to send events through log4j

[viranis]

I have setup the trial version of Splunk enterprise on my machine and have also created a dummy java spring boot service with log4j2 framework. The idea is to capture the logs in splunk from this service using HEC. I did find a nice tutorial that i followed but i still seem to not receive any events in splunk. 

Also, as there are no error messages that i can see, i am not sure what the issue is ? Can someone please guide me to the right place ?

This is the guide that i followed : https://github.com/devadyuti/integration-repo/tree/master/spring-log4j2-splunk

Please let me know if there is anything else i can provide that would be useful. 

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
        <modelVersion>4.0.0</modelVersion>
        <parent>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-parent</artifactId>
                <version>2.4.0</version>
                <relativePath/> <!-- lookup parent from repository -->
        </parent>
        <groupId>com.example</groupId>
        <artifactId>splunk-log4j</artifactId>
        <version>0.0.1-SNAPSHOT</version>
        <name>splunk-log4j</name>
        <description>Demo project for Splunk with springboot</description>

        <properties>
                <java.version>11</java.version>
        </properties>

        <repositories>
                <repository>
                        <id>splunk-artifactory</id>
                        <name>Splunk Releases</name>
                        <!--<url>https://splunk.artifactoryonline.com/artifactory/ext-releases-local</url>-->
                        <url>https://splunk.jfrog.io/splunk/ext-releases-local</url>
                </repository>
        </repositories>

        <dependencies>
                <dependency>
                        <groupId>org.springframework.boot</groupId>
                        <artifactId>spring-boot-starter-web</artifactId>
                        <exclusions>
                                <exclusion>
                                        <groupId>org.springframework.boot</groupId>
                                        <artifactId>spring-boot-starter-logging</artifactId>
                                </exclusion>
                        </exclusions>
                </dependency>

                <dependency>
                        <groupId>org.springframework.boot</groupId>
                        <artifactId>spring-boot-starter-test</artifactId>
                        <scope>test</scope>
                </dependency>

                <dependency>
                        <groupId>org.springframework.boot</groupId>
                        <artifactId>spring-boot-starter-log4j2</artifactId>
                </dependency>

                <dependency>
                        <groupId>org.springframework.boot</groupId>
                        <artifactId>spring-boot-starter-actuator</artifactId>
                </dependency>

                <dependency>
                        <groupId>com.splunk.logging</groupId>
                        <artifactId>splunk-library-javalogging</artifactId>
                        <version>1.8.0</version>
                </dependency>
        </dependencies>

        <build>
                <plugins>
                        <plugin>
                                <groupId>org.springframework.boot</groupId>
                                <artifactId>spring-boot-maven-plugin</artifactId>
                        </plugin>
                </plugins>
        </build>

</project>

log4j2.xml

<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
    <Appenders>
        <Console name="console" target="SYSTEM_OUT">
            <PatternLayout
                    pattern="%style{%d{ISO8601}} %highlight{%-5level }[%style{%t}{bright,blue}] %style{%C{10}}{bright,yellow}: %msg%n%throwable" />
        </Console>
        <SplunkHttp
            name="splunkhttp"
            url="http://127.0.0.1:8000/services/collector/event"
            token="xxxxxxxxxxxxxxxxxxxxxx"
            index="http_log_event_collector_idx"
            host="127.0.0.1"
            type="raw"
            sourcetype="_json"
            messageFormat="text"
            disableCertificateValidation="true"
            >
            <PatternLayout pattern="%m" />
        </SplunkHttp>

    </Appenders>

    <Loggers>
        <!-- LOG everything at INFO level -->
        <Root level="trace">
            <AppenderRef ref="console" />
            <AppenderRef ref="splunkhttp" />
        </Root>
    </Loggers>
</Configuration>

[viranis]

I think i have gotten it to work now. One of the issues i see is that in splunk, i had selected the source type as _json. But as soon as i changed it to log4j, i saw the logs. I will play around with this but the 8088 did the trick as well. Not sure how i missed that. Thanks

[richgalloway · Accepted solution]

The HEC port number usually is 8088 rather than 8000.  Correct the url in the log4j.xml and see if you get results.

[viranis]

Unfortunately, that did not work. I have attached the screenshot from splunk enterprise