Deployment Architecture

Universal Forwarder is slow to manage large files

tomk1
Engager

Hello,
I use an Universal Forwarder to monitor syslog-ng logs. The logs are splited in 24 logs for one day (so 1 log per hour). Each size of the log is between 300 and 600 MB, the log are sent with 5 hours of lag but they should be forwarded to index over time. The problem is the Universal Forwarder is very slow to send these logs. I quickly have behind (I receive mor log than I send). I cutomised my configuration thanks to this article : https://docs.splunk.com/Documentation/Splunk/7.3.0/Troubleshooting/Troubleshootingeventsindexingdela...

I put the limits.conf in my app package like that :

[thruput]
maxKBps = 4096

server.conf :

[queue=parsingQueue]
maxSize = 10MB

I use Splunk Universal Forwarder 7.0.8, I don't have control of indexer (But please, not that is the thruput which can't be improved and I am pretty sure that the problem is not the indexer)
I use it but the problem was already here before enable it.
I also tryied with 1, 2 and 10 pipeline and the problem persists. The thruput is capped at the equivalent of 512 KBPS. I don't have any idea about the cause of the problem,I read a lot of forum and documentation but nothing solve it. How can I investigate on the problem (my UF is running under RedHat 7). Thanks.

Thanks.

dhanasekvi
Engager

I have the same problem too. 
Is there any solution identified ?

0 Karma

akshatj2
Path Finder

were you able to find a solution for this? how did you improve performance for your UF

0 Karma
Get Updates on the Splunk Community!

Dashboard Studio Challenge - Learn New Tricks, Showcase Your Skills, and Win Prizes!

Reimagine what you can do with your dashboards. Dashboard Studio is Splunk’s newest dashboard builder to ...

Introducing Edge Processor: Next Gen Data Transformation

We get it - not only can it take a lot of time, money and resources to get data into Splunk, but it also takes ...

Take the 2021 Splunk Career Survey for $50 in Amazon Cash

Help us learn about how Splunk has impacted your career by taking the 2021 Splunk Career Survey. Last year’s ...