Monitoring Splunk

Does the size of a log file impact performance?

aafogles
Explorer

I'm running on a system with specs lower than they should be, particularly in the RAM department, (which I plan on fixing) but in the meantime, is there any benefit in reducing the size of a log file that currently is allowed to grow to 2GB before being rolled over? Thanks!

0 Karma
1 Solution

lguinn2
Legend

Not really.

However, if you are asking Splunk to monitor a directory, you can generally improve performance by cleaning out old, dead files from the directory. When you tell Splunk to "monitor" a directory, it has to keep checking all the files to make sure they haven't changed. I've seen folks point Splunk at a directory tree with over 15K files in it - a real waste of resources if only a couple of hundred files are actually being updated...

Probably not an issue on most Splunk indexers, but comes up fairly often on Splunk forwarders.

View solution in original post

lguinn2
Legend

Not really.

However, if you are asking Splunk to monitor a directory, you can generally improve performance by cleaning out old, dead files from the directory. When you tell Splunk to "monitor" a directory, it has to keep checking all the files to make sure they haven't changed. I've seen folks point Splunk at a directory tree with over 15K files in it - a real waste of resources if only a couple of hundred files are actually being updated...

Probably not an issue on most Splunk indexers, but comes up fairly often on Splunk forwarders.

Get Updates on the Splunk Community!

Community Content Calendar, November Edition

Welcome to the November edition of our Community Spotlight! Each month, we dive into the Splunk Community to ...

October Community Champions: A Shoutout to Our Contributors!

As October comes to a close, we want to take a moment to celebrate the people who make the Splunk Community ...

Stay Connected: Your Guide to November Tech Talks, Office Hours, and Webinars!

What are Community Office Hours? Community Office Hours is an interactive 60-minute Zoom series where ...