topic Sending firewall syslogs to multiple splunk servers? in Getting Data In

Sending firewall syslogs to multiple splunk servers?

yuwtennis — Fri, 11 Oct 2013 11:49:52 GMT

Hi!

We have a environment which has two HA none clustered splunk servers.
We are considering to gather firewall logs but the firewall can send to
only single splunk server. To send it to splunk servers as data cloning method,
we might need to set up intermediate forward servers .

Would this be a correct option?
Any idea would be appreciated!

Thanks,
Yu

Re: Sending firewall syslogs to multiple splunk servers?

MHibbin — Fri, 11 Oct 2013 12:04:56 GMT

I would set up a forwarder on some utility server somewhere near the firewalls, and configure the outputs.conf file to send to your two indexer.

You will need to make sure you turn off the autoLB feature in the outputs.conf file to stop the forwarder switching between the two.

Outputs.conf.spec:
"http://docs.splunk.com/Documentation/Splunk/6.0/Admin/Outputsconf"

Example configuration on UF:

inputs.conf:

[udp://514]
disabled = false
sourcetype = syslog
connection_host = none

outputs.conf:

[tcpout]
defaultGroup=indexers

[tcpout:indexers]
server=<indexer1>:9997, <indexer2>:9997
autoLB=false

Restart of Splunkd on the forwarder.

Hope this helps,

MHibbin

Re: Sending firewall syslogs to multiple splunk servers?

alacercogitatus — Fri, 11 Oct 2013 12:14:28 GMT

If I'm reading this correctly, you want the data available on both indexers for HA purposes, but don't use clustering. You want "index replication" without enabling clustering. Correct?

Re: Sending firewall syslogs to multiple splunk servers?

yuwtennis — Fri, 11 Oct 2013 12:18:31 GMT

Hi!

Thank you for the reply.

Yes that is correct.

Re: Sending firewall syslogs to multiple splunk servers?

yuwtennis — Fri, 11 Oct 2013 12:21:15 GMT

Hi!

Thanks for the comment.

Thtas what I thought too.

Thanks.

Re: Sending firewall syslogs to multiple splunk servers?

alacercogitatus — Fri, 11 Oct 2013 12:28:53 GMT

MHibbin is close. The configuration he has will randomly pick an indexer and send it there, but only to one until the connection is reset. If you want to clone the data across two indexers, use this config:

outputs.conf

[tcpout]

[tcpout:indexer1]
server = :9997

[tcpout:indexer2]
server = :9997

Ref: http://docs.splunk.com/Documentation/Splunk/5.0.4/Admin/Outputsconf#outputs.conf.example

Re: Sending firewall syslogs to multiple splunk servers?

kristian_kolb — Fri, 11 Oct 2013 12:41:47 GMT

What is the difference between this configuration and the following?

[tcpout]
defaultGroup = primary, secondary

[tcpout:primary]
server = 10.11.12.13:9997

[tcpout:secondary]
server = 14.15.16.17:9997

From the outputs.conf docs.
You can have as many target groups as you want.
If more than one target group is specified, the forwarder sends all data to each target group.
This is known as "cloning" the data.

and

autoLB = true * Automatic load balancing is the only way to forward data. Round-robin method is not supported anymore. * Defaults to true.

Re: Sending firewall syslogs to multiple splunk servers?

alacercogitatus — Fri, 11 Oct 2013 12:56:57 GMT

server = idx1:9997, idx2:9997 within a single group indicates a loadbalance, but if you autoLB false, then it sends to the first one it hits. Using two groups says "send to all groups, no Load balancing."

Re: Sending firewall syslogs to multiple splunk servers?

kristian_kolb — Sat, 12 Oct 2013 08:27:41 GMT

That's pretty close to what I thought, alacer.