Deployment Architecture

Universal Forwarder is slow to manage large files

tomk1
Engager

Hello,
I use an Universal Forwarder to monitor syslog-ng logs. The logs are splited in 24 logs for one day (so 1 log per hour). Each size of the log is between 300 and 600 MB, the log are sent with 5 hours of lag but they should be forwarded to index over time. The problem is the Universal Forwarder is very slow to send these logs. I quickly have behind (I receive mor log than I send). I cutomised my configuration thanks to this article : https://docs.splunk.com/Documentation/Splunk/7.3.0/Troubleshooting/Troubleshootingeventsindexingdela...

I put the limits.conf in my app package like that :

[thruput]
maxKBps = 4096

server.conf :

[queue=parsingQueue]
maxSize = 10MB

I use Splunk Universal Forwarder 7.0.8, I don't have control of indexer (But please, not that is the thruput which can't be improved and I am pretty sure that the problem is not the indexer)
I use it but the problem was already here before enable it.
I also tryied with 1, 2 and 10 pipeline and the problem persists. The thruput is capped at the equivalent of 512 KBPS. I don't have any idea about the cause of the problem,I read a lot of forum and documentation but nothing solve it. How can I investigate on the problem (my UF is running under RedHat 7). Thanks.

Thanks.

verbal_666
Builder

Hi.

I also have a large size files on some servers, about 10Gb per day in 3 files each server, and those files during the day are very delayed to be ingested, with ACK to true.

While those files delay from 1 to also 4 hours to be indexed, other files on same servers are ingested fine in realtime.

So, also with UF 8.2.12, i think it's a thruput of Network Infrastructure, or maybe too many datas from those inputs 🤷‍♂️

I also have

 

[thruput]
maxKBps = 0

[general]
parallelIngestionPipelines = 2
[queue]
maxSize = 100MB
[queue=parsingQueue]
maxSize = 10MB

I don't think there are other methods, since it's a phisiological problem 🤷‍♂️

The only way, maybe, is to add more Indexers in SPLUNK Infra or ask the Applicative Teams to split those file in more servers 🤷‍♂️

0 Karma

dhanasekvi
Engager

I have the same problem too. 
Is there any solution identified ?

0 Karma

akshatj2
Path Finder

were you able to find a solution for this? how did you improve performance for your UF

0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

March Community Office Hours Security Series Uncovered!

Hello Splunk Community! In March, Splunk Community Office Hours spotlighted our fabulous Splunk Threat ...

Stay Connected: Your Guide to April Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars in April. This post ...