Deployment Architecture

How can I increase the universal forwarder buffer size to insure logs are not lost when site becomes isolated?

nickcc
Engager

The network connectivity at one of my sites drops on occasions. So, I was reading best practice is to configure a syslog collector to ingest logs via UDP 514 at local site and forward those to an indexer offsite via universal forwarder TCP 9997.

Now, if the site loses network connectivity and the collector no longer has the capability to forward logs to the indexer, will the universal forwarder (on collector) buffer the logs to make certain nothing is missed once everything comes up and its able to send logs to the indexer? I saw that the use persistent queues are not available for specific types and splunktcp (input from Splunk forwarder) was on the list.

My question is, how can I increase the buffer size (if all) to insure my logs are not lost when my site becomes isolated. Also if I have this completely wrong and there is a better way, please let me know. Any assistance would be greatly appreciated

0 Karma
1 Solution

nickhills
Ultra Champion

The recommended approach is to send Syslog messages to a syslog receiver like syslog-ng.
The syslog receiver will write the events to file, and then you use a forwarder to collect the results from file, and send to your indexers.

This breaks the "real-time" dependency for syslog, and means that you can tolerate network disruption between the UF-indexers, rather than risk loosing syslog messages in flight.

If my comment helps, please give it a thumbs up!

View solution in original post

nickhills
Ultra Champion

The recommended approach is to send Syslog messages to a syslog receiver like syslog-ng.
The syslog receiver will write the events to file, and then you use a forwarder to collect the results from file, and send to your indexers.

This breaks the "real-time" dependency for syslog, and means that you can tolerate network disruption between the UF-indexers, rather than risk loosing syslog messages in flight.

If my comment helps, please give it a thumbs up!
Get Updates on the Splunk Community!

BSides Splunk 2022 - The Call for Papers is now Open!

TLDR; Main Site: https://bsidessplunk.com CFP Site: https://bsidessplunk.com/cfp CFP Opens: December 15th, ...

Sending Metrics to Splunk Enterprise With the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

What's New in Splunk Cloud Platform 9.0.2208?!

Howdy!  We are happy to share the newest updates in Splunk Cloud Platform 9.0.2208! Analysts can benefit ...