Using Java app with Splunk - Commands, scripts or modular inputs?

[alekksi]

Hi all,

I've been having difficulty trying to set up Splunk to use a simple Java app I have created to feed data into Splunk from multiple Windows (x86 and x86_64) servers. The app itself is rather straightforward so it's not worth talking that much about excluding its execution, which includes one parameter:

java -jar SplunkTest.jar start

I've been reading up on modular inputs (Howto, ModInputsSpec & ModInputsExample) which sound like it could be the solution to my problem, but it doesn't appear to be straightforward how to do this. For example, I'm not sure where the application is invoked from, nor how I would set the interval.

Using commands.conf seems promising as well, but the examples are only for perl/python. I would prefer not to use a python wrapper to invoke a java application, but this could work as well. Scripts I guess fall under a similar category.

I've tried to set up my installation as per the first howto link above, but I'm still at a bit of a loss. It currently looks like this:

default/

-> app.conf

-> inputs.conf (empty)

jars/

-> SplunkTest.jar

local/

metadata/

-> default.meta

windows_x86/bin/

-> SplunkTest.exe (renamed from shim-windows_x86.exe)

windows_x86_64/bin/

-> SplunkTest.exe (renamed from shim-windows_x86_64.exe)

Has anyone else tried to do this or has a successful implementation of similar? This framework, for example, served just to confuse me more than help me.

Any help would be greatly appreciated!!

Regards,

Alex

[alekksi]

In a way, it's kind of irrelevant what my data source is. I have a java app that generates data from a source, then connects via a TCP socket to the local SplunkForwarder, which then forwards data to the collectors.

As you asked, the datasource is a CSV file generated by Windows Performance Monitor. As this is a messy file, it can't be sent directly to Splunk without formatting the data, also out-of-the-box Splunk PerfMon app is not fit for purpose.

[Damien_Dallimor]

Can you provide some details about the data source you are trying to get into Splunk ? Log files ? In flight App data ? Data from some API ?

The experts on this forum will then be able to advise of the best way to get this data into Splunk.

[alekksi · Accepted solution]

My solution isn't great, but it does work. Here it is below for anyone that wants:

inputs.conf

[script://.\bin\splunktest.bat]

index=default

interval=60

sourcetype = splunk_test

source = splunk_test

disabled = 0

splunktest.bat

@echo off

set splunktest=%SPLUNK_HOME%\etc\apps\SplunkTest\jars

cd /d %splunktest%

java -jar splunktest.jar

Edits: Formatting and the like

[derekarnold]

You have a couple of options. It seems you want to get logs from some sort of homegrown Java app into Splunk. Could you not write the relevant output to a standard log file then send it to Splunk using universal forwarder? Is that an option for you? You could have SplunkTest.jar log to output.txt using out.println or whatever, then have an app set up that monitors the directory containing the log file.

Apart from that you could also consider using a wrapper script to invoke your application. I know it could seem a bit wrong but I have successfully done this in the past.

[alekksi]

No, unfortunately not. This SplunkTest app is just to check that I can invoke Java apps using Splunk. The actual end-goal is to use my own PerfMon:// csv files with my own specifications, as I've found the Splunk for Windows Infrastructure app massively lacking, by which I mean it accounts for a third of all our total data usage, when it is only monitoring a small number of servers, in comparison to our Linux stats. The Java app I have written basically keeps all the counters for a given object+instance pair on a single line, using less data and being more searchable.