"Secure Data Pipeline" with Splunk

[beaunewcomb]

I'm working on a POC with devs of a web application and we want to send personally identifiable information across our network into Splunk. The concept is 1, get the forwarder as close to the application as possible, 2, avoid writing to disk anywhere other than on the indexer (which is being written encrypted, but that's taken care of), and 3, use the forwarder to encrypt data in flight.

I was thinking about setting up the forwarder listening on a network port, using iptables to restrict access, and have the app log out via network socket right into the forwarder.

Any thoughts on this? I want to avoid writing to disk of possible, but also want to ensure we don't miss any events. I want some way to hand off events straight from the app to the forwarder, making sure the forwarder is actually running and taking events.

[kristian_kolb]

That would be an...odd way to go about it, and I'm not sure it would work. You can configure the Forwarder and Indexer to SSL-encrypt communications for log transport.

Or look into @dart's recommendation.

[dart]

You could use a modular input for this, or have your application post directly to Splunk, via the REST endpoint
http://docs.splunk.com/Documentation/Splunk/latest/RESTAPI/RESTinput#receivers.2Fsimple

[brettcave]

what about using SSH tunneling? forwarder -> localhost:2220 -> ssh tunnel -> indexer:index_port

in some of our application components, we use a syslog appender to send data from the app to the forwarder (udp port 514 listener), and if you put in a ssh tunnel between forwarder and indexer you should have secure data.

[kristian_kolb]

Well, using syslog over TCP will give you some assurance. I guess that you could install the forwarder locally on the app-server, and set it to listen on e.g. localhost:1514. The app could then log there i.e. not sending stuff out on the network at all.

Then set up SSL for the forwarder->indexer traffic.

Haven't tried it, but it should work.