Getting Data In

Is there a maximum number of syslog events that forwader can receive directly?

Takajian
Builder

I will set up AutoLB on intermediate forwarder. The syslog event from many network devices will be sent to the forwarder and it forward the data to multiple indexers. The syslog data size will be over 10 GB per day. If there is a maximum number of syslog events that single forwarder can receive? I understand if I use syslog-ng and monitor the syslog text file, I do not care about this. However I have to use windows environment, so I can not use syslog-ng, the forwarder need to receive syslog directly.

Tags (1)
0 Karma

stuartamurray
Path Finder

Hmm this is not necessarily true.

The indexer can for sure take 10GB per day easily but you may have an issue if your forwarders are trying to push that amount as they are limited. Depends what your peak traffic rate is.

By default a lightweight forwarder is limited to 256KBps or around 15MBph (thats Bytes not bits). So you may find your forwarders getting behind.

I've seen instances where my forwarders are 8 hours behind, although with much chunkier log sizes.

My solution has been a combination of: Either run as a standard rather than lightweight forwarder Comment out the maxKBps directive in limits.conf (has the advantage of running without web) Reduce the log size produced by that node.

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

There is no maximum number, and 10GB/day is a very low rate. Of course, if you have extremely high intra-day peaks, the network, network stack, or OS may drop packets.

Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...