Deployment Architecture

load balancer

hazem
Path Finder

Is there any documentation in Splunk's documentation to guide a load balancer administrator on configuring the load balancer in front of intermediate forwarders to receive syslog traffic from security devices on port 514?

Labels (1)
0 Karma
1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi @hazem ,

as I said,

On the LB you have only to configure the rule to associate the receiving port with the ip addresses and port of the receivers.

In addition, depending on the LB, you should configure how the LB checks if the receivers are alive, but this isn't a Splunk configuration and it depends on the LB (and I cannot help you.

In other word: you must define a VIP and a port to use to send logs from the syslog sources, and then associate these VIP and port to the destination IP addresses and port (of the UFs.

There isn't a best practice, only that the LB must check if the destinations are alive.

There's only one not clear thing: why are you speaking of a single intermediate Forwarder?

To have HA, you need at least two UFs, otherwise the LB is completely useless.

Ciao.

Giuseppe

View solution in original post

0 Karma

hazem
Path Finder

Dear @gcusello 

I have already configured rsyslog on both intermediate forwarders and need to set up the load balancer to receive traffic from syslog devices and forward it to a single backend intermediate forwarder. If the load balancer administrator asks, what is the best practice for configuring the load balancer to forward traffic to our intermediate forwarder?

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @hazem ,

as I said,

On the LB you have only to configure the rule to associate the receiving port with the ip addresses and port of the receivers.

In addition, depending on the LB, you should configure how the LB checks if the receivers are alive, but this isn't a Splunk configuration and it depends on the LB (and I cannot help you.

In other word: you must define a VIP and a port to use to send logs from the syslog sources, and then associate these VIP and port to the destination IP addresses and port (of the UFs.

There isn't a best practice, only that the LB must check if the destinations are alive.

There's only one not clear thing: why are you speaking of a single intermediate Forwarder?

To have HA, you need at least two UFs, otherwise the LB is completely useless.

Ciao.

Giuseppe

0 Karma

isoutamo
SplunkTrust
SplunkTrust
As I said, this depends on LB. In some vendors you will lose event if LB check if backend is up or down, and if it is down then e.g. F5 just drop the packet (or at least it done it couple of years ago when I last use it). You must change the profile to get it to work correctly.
0 Karma

hazem
Path Finder

Hello @gcusello 

regarding your question:

There's only one not clear thing: why are you speaking of a single intermediate Forwarder?

No, I have 2 forwarders, but as you know, since UDP is a stream, one forwarder will handle all traffic.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @hazem ,

good for you, see next time!

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated by all the contributors 😉

0 Karma

hazem
Path Finder

Hello @gcusello 

please advise the load balancing method connectivity from source IP to go to the forwarder node .. the persistency or we can keep it round robin?

we have 2 forwarders

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @hazem ,

I'm not an exper in Load Balancers, and, as @isoutamo said, it depends on the Load Balancer: ask this question to a specialist of your LB.

Ciao.

Giuseppe.

P.S.: Karma Points are appreciated by all the contributors 😉

0 Karma

PickleRick
SplunkTrust
SplunkTrust

Actually, UDP is _not_ a stream. UDP is a connectionless protocol and every datagram is independent from all other ones.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @hazem ,

ok, not it's clear.

Anyway, 

let us know if we can help you more, or, please, accept one answer for the other people of Community.

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated by all the contributors 😉

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @hazem ,

adding a bit to @PickleRick information:

you can configure rsyslog (or syslog-ng) server on your UFs: you don't need to install it because it's already installe, you have only to configure it to understand where tp write logs.

for more infos see at https://www.rsyslog.com/guides/ or https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/6/html/deployment_guide/s1-basic_c... 

on the LB, you need only to configure the receiving port and the destination port and addresses.

Some LBs need also to configure a way to check if the destinations are alive, but this configuration depends on your LB and it's indipendent by Splunk or rsyslog receiver.

Ciao.

Giuseppe

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Splunk recommendation is to NOT send syslog data directly (or via a LB) to a Splunk instance.  Syslog should be sent to a dedicated syslog server (running syslog-ng or rsyslog) and then forwarded to Splunk.  The syslog servers should be positioned as close to the data source as possible to avoid data loss.  Use of a load balancer in front of the syslog servers is recommended for resiliency.

For more information, see

https://docs.splunk.com/Documentation/Splunk/latest/Data/HowSplunkEnterprisehandlessyslogdata#Caveat...

https://www.splunk.com/en_us/blog/tips-and-tricks/high-performance-syslogging-for-splunk-using-syslo...

 

---
If this reply helps you, Karma would be appreciated.
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Let me disagree here with you on one thing. Adding a load balancer in front of syslog receivers does not usually solve any problems (especially because LBs typically "don't speak" syslog; and even more so since "syslog" can mean many different things - from RFC5424-compliant message to "just throw anyting at UDP/514") and introduces additional layer of complexity and a potential SPOF.

0 Karma

isoutamo
SplunkTrust
SplunkTrust
That's true when you are using UDP and pure syslog. But fortunately usually you can select other transform protocol that UDP for syslog or even use rsyslog with RELP. In those cases I strongly recommend to use LB with correct configurations.
0 Karma

hazem
Path Finder

hello @isoutamo 


please advise the load balancing method connectivity from source IP to go to the forwarder node .. the persistency or we can keep it round robin?

we have 2 forwarders

0 Karma

isoutamo
SplunkTrust
SplunkTrust
As I said, this depends on e.g. are you using plain syslog, udp vs tcp vs tsl etc.
If you are using udp there is no matter what you are doing, you will lose some events anyway.
0 Karma

PickleRick
SplunkTrust
SplunkTrust

There is no such document because generally it's not recommended to LB "syslog" traffic.

You should keep your syslog receiver as simple as possible and as close to the source as possible.

0 Karma

isoutamo
SplunkTrust
SplunkTrust

As most other configuration items this also has answer it depends on your environment. There are some known issues with some vendors and as @PickleRick said, pure syslog traffic with UDP is not good candidate for LB. But e.g. if you are using rsyslog with RELP protocol then it's totally different case. You could use e.g. F5 in front of rsyslog backends and this works well after you have select e.g. FastL4 profile for LB. Without it you will lost some events.

r. Ismo

0 Karma
Get Updates on the Splunk Community!

New Case Study Shows the Value of Partnering with Splunk Academic Alliance

The University of Nevada, Las Vegas (UNLV) is another premier research institution helping to shape the next ...

How to Monitor Google Kubernetes Engine (GKE)

We’ve looked at how to integrate Kubernetes environments with Splunk Observability Cloud, but what about ...

Index This | How can you make 45 using only 4?

October 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...