topic HowTo pull logs into trusted network from a forwarder located in DMZ in Getting Data In

HowTo pull logs into trusted network from a forwarder located in DMZ

tjensen — Mon, 05 Nov 2012 10:21:32 GMT

Hello,
I search a way to get realtime logs from DMZ-Zone into a Trusted Network, where the Indexer is located.
A Forwarder located in DMZ collects all logs, but this Forwarder can only send (push) logs to the trusted network. Is there a way to change the direction of the communication to pull these logs from the Forwarder?

I can pull all Logfiles directly from the log-source, but this is not in realtime.

Any suggestions on this ?

Thanks,
Torsten

Re: HowTo pull logs into trusted network from a forwarder located in DMZ

lrhazi — Mon, 05 Nov 2012 13:39:41 GMT

I dont know if Splunk indexer-forwarder supports the feature you need, but a workaround might be to to use something like stunnel or OpenSSH, to create a tunnel that is "listening" on the forwarder, and "forwarding" to the indexer. Your forwarder would then be configured to forward to localhost:port.

With SSH this would be called reverse tunnel and would be something like:

On the indexer: ssh -R 6514:localhost:6514 username@forwarder

Now on the forwarder, if you connect to localhost:6514, you would be connected to the indexer:6514

Re: HowTo pull logs into trusted network from a forwarder located in DMZ

bmacias84 — Mon, 05 Nov 2012 17:44:50 GMT

What you are trying to accomplish does not exist in Splunk nativly, but there is always a method. You could use rsync with following switches --stats -rltgoDzrv --append-verify to copy the data to Trusted Network Forward and setup a Monitor on that directory.



rsync --stats -rltgoDzrv --append-verify -e "ssh -l ssh-user" rsync:://targethost2/module/src/ /tmp/secure_data/

Or Possibly a scripted input using ssh and tail.

Re: HowTo pull logs into trusted network from a forwarder located in DMZ

tjensen — Tue, 06 Nov 2012 07:44:57 GMT

Thanks for this feedback. We'll try it like this way.

Re: HowTo pull logs into trusted network from a forwarder located in DMZ

vial8 — Tue, 06 Nov 2012 08:08:27 GMT

Definately the best way to go. I use this on our solution where we have logs that is on a vendor system.