Archive

Universal Forwarder is slow to manage large files

tomk1
Engager

Hello,
I use an Universal Forwarder to monitor syslog-ng logs. The logs are splited in 24 logs for one day (so 1 log per hour). Each size of the log is between 300 and 600 MB, the log are sent with 5 hours of lag but they should be forwarded to index over time. The problem is the Universal Forwarder is very slow to send these logs. I quickly have behind (I receive mor log than I send). I cutomised my configuration thanks to this article : https://docs.splunk.com/Documentation/Splunk/7.3.0/Troubleshooting/Troubleshootingeventsindexingdela...

I put the limits.conf in my app package like that :

[thruput]
maxKBps = 4096

server.conf :

[queue=parsingQueue]
maxSize = 10MB

I use Splunk Universal Forwarder 7.0.8, I don't have control of indexer (But please, not that is the thruput which can't be improved and I am pretty sure that the problem is not the indexer)
I use it but the problem was already here before enable it.
I also tryied with 1, 2 and 10 pipeline and the problem persists. The thruput is capped at the equivalent of 512 KBPS. I don't have any idea about the cause of the problem,I read a lot of forum and documentation but nothing solve it. How can I investigate on the problem (my UF is running under RedHat 7). Thanks.

Thanks.

dhanasekvi
Engager

I have the same problem too. 
Is there any solution identified ?

0 Karma

akshatj2
Path Finder

were you able to find a solution for this? how did you improve performance for your UF

0 Karma
.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!