Can I inject events into Splunk via a script?

[Justin_Grant]

Splunk supports scripted inputs (where splunk calls a script and indexes the results). But what about the reverse: can I run a script from outside Splunk to inject events into Splunk?

The particular case I'm thinking of is correlating alerts produced by another management tool (e.g. SiteScope or Tivoli) with data already stored in Splunk. The other management tools have the ability to run a command-line program or script in response to an alert, but what program should I call in order to inject data into Splunk?

Another case is manually reporting human status into splunk. For example, imagine if I could type this at the command line:

SPLUNK INJECT --sourcetype=manual --host=WEB01 --user=jgrant "Rebooting WEB01 now, ignore alerts for 5 mins"

I realize there are workarounds here, like piping the output of the script into a directory and having splunk index that directory, but I'm wondering if there's a direct connection possible without a directory as an intermediary.

[jrodman]

If going of the network is somehow inconvenient (not a fan of nc), you can also set up a FIFO (named pipe) input and write your data to this. This has the positive that the flow control will give you a good idea when the data has been fully accepted. It has of course the negative of all transient data interfaces that it will be difficult to review in case of a problem.

This is most appropriate when your source data is relatively persistent so you can handle discontinuities in service.

[gkanapathy]

Other things you can do is, if you have a UDP or TCP input set up on your indexer, just use ncat or nc to send a line or a whole file to Splunk. Windows and Linux and Mac. e.g UDP:

echo "blah blah blah" | ncat -u splunkhost 514

ncat -u < myfileinput.log

ncat for Windows and Mac is at http://nmap.org/ncat/

nc is builtin on Linux (also on Mac, but doesn't seem to send UDP).

[ampledata]

you could also try this from the command line:

echo "Rebooting WEB01 now, ignore alerts for 5 mins" | curl  -d "sourcetype=manual" -d "host=WEB01" -k -u admin:changeme https://splunk.example.com:8089/services/receivers/stream

[ampledata]

if this is a UNIX system you can use logger(1) to log a message using syslog:

logger "sourcetype=manual host=WEB01 user=jgrant 'Rebooting WEB01 now, ignore alerts for 5 mins'"

in your /etc/syslog.log add a line as follows:

*.* @splunk.example.com

and in splunk.example.com add a udp input on port 514.

[Justin_Grant]

I'm on a Windows box today so I'll go with @Johnvey's suggestion above, but this is a great suggestion for non-Windows. +1!

[Johnvey · Accepted solution]

Splunk 4.0 has an experimental input endpoint located at:

http://YOUR_SPLUNKD_HOST:8089/services/receivers/stream

which takes a POST request. To submit your example code, the HTTP request looks like:

POST /services/receivers/stream?sourcetype=manual&host=WEB01 HTTP/1.0
Host: YOUR_SPLUNKD_HOST
Content-Length: 46
Content-Type: text/plain; charset=utf-8

Rebooting WEB01 now, ignore alerts for 5 mins

The entire body of the POST will be treated as the raw event and will be sent through the classifier, aggregator, timestamper, and typer.

The built-in python SDK has methods around this, located at:

$SPLUNK_HOME/lib/python2.6/site-packages/splunk/input.py

On a desktop machine, the new event will be available in the index within 20 seconds or so; within 3 seconds if running a real-time search beforehand.

[mahbe]

I tried this but the input via curl doesn't show up in Splunk.

C:>curl -d @foobar http://localhost:8089/services/receivers/stream
curl: (52) Empty reply from server

Do I have to enable a special datasource in Splunk? I use the latest version of Splunk.

[Justin_Grant]

Dude, that rocks. Exactly what I was looking for. Accepted!