Getting Data In

Sending firewall syslogs to multiple splunk servers?

yuwtennis
Communicator

Hi!

We have a environment which has two HA none clustered splunk servers.
We are considering to gather firewall logs but the firewall can send to
only single splunk server. To send it to splunk servers as data cloning method,
we might need to set up intermediate forward servers .

Would this be a correct option?
Any idea would be appreciated!

Thanks,
Yu

0 Karma
1 Solution

MHibbin
Influencer

I would set up a forwarder on some utility server somewhere near the firewalls, and configure the outputs.conf file to send to your two indexer.

You will need to make sure you turn off the autoLB feature in the outputs.conf file to stop the forwarder switching between the two.

Outputs.conf.spec:
"http://docs.splunk.com/Documentation/Splunk/6.0/Admin/Outputsconf"

Example configuration on UF:

inputs.conf:

[udp://514]
disabled = false
sourcetype = syslog
connection_host = none

outputs.conf:

[tcpout]
defaultGroup=indexers

[tcpout:indexers]
server=<indexer1>:9997, <indexer2>:9997
autoLB=false

Restart of Splunkd on the forwarder.

Hope this helps,

MHibbin

View solution in original post

alacercogitatus
SplunkTrust
SplunkTrust

MHibbin is close. The configuration he has will randomly pick an indexer and send it there, but only to one until the connection is reset. If you want to clone the data across two indexers, use this config:

outputs.conf


[tcpout]

[tcpout:indexer1]
server = :9997

[tcpout:indexer2]
server = :9997

Ref: http://docs.splunk.com/Documentation/Splunk/5.0.4/Admin/Outputsconf#outputs.conf.example

yuwtennis
Communicator

Hi!

Thank you for the reply.

Yes that is correct.

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

If I'm reading this correctly, you want the data available on both indexers for HA purposes, but don't use clustering. You want "index replication" without enabling clustering. Correct?

0 Karma

MHibbin
Influencer

I would set up a forwarder on some utility server somewhere near the firewalls, and configure the outputs.conf file to send to your two indexer.

You will need to make sure you turn off the autoLB feature in the outputs.conf file to stop the forwarder switching between the two.

Outputs.conf.spec:
"http://docs.splunk.com/Documentation/Splunk/6.0/Admin/Outputsconf"

Example configuration on UF:

inputs.conf:

[udp://514]
disabled = false
sourcetype = syslog
connection_host = none

outputs.conf:

[tcpout]
defaultGroup=indexers

[tcpout:indexers]
server=<indexer1>:9997, <indexer2>:9997
autoLB=false

Restart of Splunkd on the forwarder.

Hope this helps,

MHibbin

kristian_kolb
Ultra Champion

That's pretty close to what I thought, alacer.

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

server = idx1:9997, idx2:9997 within a single group indicates a loadbalance, but if you autoLB false, then it sends to the first one it hits. Using two groups says "send to all groups, no Load balancing."

0 Karma

kristian_kolb
Ultra Champion

What is the difference between this configuration and the following?

[tcpout]
defaultGroup = primary, secondary

[tcpout:primary]
server = 10.11.12.13:9997

[tcpout:secondary]
server = 14.15.16.17:9997

From the outputs.conf docs.
You can have as many target groups as you want.
If more than one target group is specified, the forwarder sends all data to each target group.
This is known as "cloning" the data.

and

autoLB = true
* Automatic load balancing is the only way to forward data. Round-robin method is not supported anymore.
* Defaults to true.

0 Karma

yuwtennis
Communicator

Hi!

Thanks for the comment.

Thtas what I thought too.

Thanks.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...