Monitoring Splunk

Does the size of a log file impact performance?

aafogles
Explorer

I'm running on a system with specs lower than they should be, particularly in the RAM department, (which I plan on fixing) but in the meantime, is there any benefit in reducing the size of a log file that currently is allowed to grow to 2GB before being rolled over? Thanks!

0 Karma
1 Solution

lguinn2
Legend

Not really.

However, if you are asking Splunk to monitor a directory, you can generally improve performance by cleaning out old, dead files from the directory. When you tell Splunk to "monitor" a directory, it has to keep checking all the files to make sure they haven't changed. I've seen folks point Splunk at a directory tree with over 15K files in it - a real waste of resources if only a couple of hundred files are actually being updated...

Probably not an issue on most Splunk indexers, but comes up fairly often on Splunk forwarders.

View solution in original post

lguinn2
Legend

Not really.

However, if you are asking Splunk to monitor a directory, you can generally improve performance by cleaning out old, dead files from the directory. When you tell Splunk to "monitor" a directory, it has to keep checking all the files to make sure they haven't changed. I've seen folks point Splunk at a directory tree with over 15K files in it - a real waste of resources if only a couple of hundred files are actually being updated...

Probably not an issue on most Splunk indexers, but comes up fairly often on Splunk forwarders.

Get Updates on the Splunk Community!

Splunk Mobile: Your Brand-New Home Screen

Meet Your New Mobile Hub  Hello Splunk Community!  Staying connected to your data—no matter where you are—is ...

Introducing Value Insights (Beta): Understand the Business Impact your organization ...

Real progress on your strategic priorities starts with knowing the business outcomes your teams are delivering ...

Enterprise Security (ES) Essentials 8.3 is Now GA — Smarter Detections, Faster ...

As of today, Enterprise Security (ES) Essentials 8.3 is now generally available, helping SOC teams simplify ...