Getting Data In

Why does it take time to transfer a gz file?

yutaka1005
Builder

In my system architecture, UF is transfering 1.8GB GZ format Compressed ifilter log(original size is 15GB) to two IDX.
However, the transfer speed is very slow. In my calculation it will take around 24 hours to send all the logs.

The value of maxKbps is set to 0, So I am thinking that expand of the GZ file cause the matter.

But may such a matter occur?

0 Karma
1 Solution

martin_mueller
SplunkTrust
SplunkTrust

Make sure you don't change limits.conf in etc/system/default, instead create one in etc/system/local or etc/apps/some_app/local - else all changes are lost on upgrade of Splunk.

View solution in original post

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Make sure you don't change limits.conf in etc/system/default, instead create one in etc/system/local or etc/apps/some_app/local - else all changes are lost on upgrade of Splunk.

0 Karma

yutaka1005
Builder

Thanks for your answer! I changed limits.conf in etc/system/local.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

UFs are capable of unpacking gzip at more than 20kb/s (1.8GB in 24h) - something else is going on.

0 Karma

yutaka1005
Builder

Thanks for your comments ! I overlooked that "maxKBps = 256" was set in "/opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/limits.conf". When I changed the setting value, the transfer was successfully done.

0 Karma
Get Updates on the Splunk Community!

Announcing the Expansion of the Splunk Academic Alliance Program

The Splunk Community is more than just an online forum — it’s a network of passionate users, administrators, ...

Learn Splunk Insider Insights, Do More With Gen AI, & Find 20+ New Use Cases You Can ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Buttercup Games: Further Dashboarding Techniques (Part 7)

This series of blogs assumes you have already completed the Splunk Enterprise Search Tutorial as it uses the ...