Getting Data In

Sending firewall syslogs to multiple splunk servers?

yuwtennis
Communicator

Hi!

We have a environment which has two HA none clustered splunk servers.
We are considering to gather firewall logs but the firewall can send to
only single splunk server. To send it to splunk servers as data cloning method,
we might need to set up intermediate forward servers .

Would this be a correct option?
Any idea would be appreciated!

Thanks,
Yu

0 Karma
1 Solution

MHibbin
Influencer

I would set up a forwarder on some utility server somewhere near the firewalls, and configure the outputs.conf file to send to your two indexer.

You will need to make sure you turn off the autoLB feature in the outputs.conf file to stop the forwarder switching between the two.

Outputs.conf.spec:
"http://docs.splunk.com/Documentation/Splunk/6.0/Admin/Outputsconf"

Example configuration on UF:

inputs.conf:

[udp://514]
disabled = false
sourcetype = syslog
connection_host = none

outputs.conf:

[tcpout]
defaultGroup=indexers

[tcpout:indexers]
server=<indexer1>:9997, <indexer2>:9997
autoLB=false

Restart of Splunkd on the forwarder.

Hope this helps,

MHibbin

View solution in original post

alacercogitatus
SplunkTrust
SplunkTrust

MHibbin is close. The configuration he has will randomly pick an indexer and send it there, but only to one until the connection is reset. If you want to clone the data across two indexers, use this config:

outputs.conf


[tcpout]

[tcpout:indexer1]
server = :9997

[tcpout:indexer2]
server = :9997

Ref: http://docs.splunk.com/Documentation/Splunk/5.0.4/Admin/Outputsconf#outputs.conf.example

yuwtennis
Communicator

Hi!

Thank you for the reply.

Yes that is correct.

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

If I'm reading this correctly, you want the data available on both indexers for HA purposes, but don't use clustering. You want "index replication" without enabling clustering. Correct?

0 Karma

MHibbin
Influencer

I would set up a forwarder on some utility server somewhere near the firewalls, and configure the outputs.conf file to send to your two indexer.

You will need to make sure you turn off the autoLB feature in the outputs.conf file to stop the forwarder switching between the two.

Outputs.conf.spec:
"http://docs.splunk.com/Documentation/Splunk/6.0/Admin/Outputsconf"

Example configuration on UF:

inputs.conf:

[udp://514]
disabled = false
sourcetype = syslog
connection_host = none

outputs.conf:

[tcpout]
defaultGroup=indexers

[tcpout:indexers]
server=<indexer1>:9997, <indexer2>:9997
autoLB=false

Restart of Splunkd on the forwarder.

Hope this helps,

MHibbin

kristian_kolb
Ultra Champion

That's pretty close to what I thought, alacer.

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

server = idx1:9997, idx2:9997 within a single group indicates a loadbalance, but if you autoLB false, then it sends to the first one it hits. Using two groups says "send to all groups, no Load balancing."

0 Karma

kristian_kolb
Ultra Champion

What is the difference between this configuration and the following?

[tcpout]
defaultGroup = primary, secondary

[tcpout:primary]
server = 10.11.12.13:9997

[tcpout:secondary]
server = 14.15.16.17:9997

From the outputs.conf docs.
You can have as many target groups as you want.
If more than one target group is specified, the forwarder sends all data to each target group.
This is known as "cloning" the data.

and

autoLB = true
* Automatic load balancing is the only way to forward data. Round-robin method is not supported anymore.
* Defaults to true.

0 Karma

yuwtennis
Communicator

Hi!

Thanks for the comment.

Thtas what I thought too.

Thanks.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...