Monitoring Splunk

Does the size of a log file impact performance?

aafogles
Explorer

I'm running on a system with specs lower than they should be, particularly in the RAM department, (which I plan on fixing) but in the meantime, is there any benefit in reducing the size of a log file that currently is allowed to grow to 2GB before being rolled over? Thanks!

0 Karma
1 Solution

lguinn2
Legend

Not really.

However, if you are asking Splunk to monitor a directory, you can generally improve performance by cleaning out old, dead files from the directory. When you tell Splunk to "monitor" a directory, it has to keep checking all the files to make sure they haven't changed. I've seen folks point Splunk at a directory tree with over 15K files in it - a real waste of resources if only a couple of hundred files are actually being updated...

Probably not an issue on most Splunk indexers, but comes up fairly often on Splunk forwarders.

View solution in original post

lguinn2
Legend

Not really.

However, if you are asking Splunk to monitor a directory, you can generally improve performance by cleaning out old, dead files from the directory. When you tell Splunk to "monitor" a directory, it has to keep checking all the files to make sure they haven't changed. I've seen folks point Splunk at a directory tree with over 15K files in it - a real waste of resources if only a couple of hundred files are actually being updated...

Probably not an issue on most Splunk indexers, but comes up fairly often on Splunk forwarders.

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Network to App: Observability Unlocked [May & June Series]

In today’s digital landscape, your environment is no longer confined to the data center. It spans complex ...

SPL2 Deep Dives, AppDynamics Integrations, SAML Made Simple and Much More on Splunk ...

Splunk Lantern is Splunk’s customer success center that provides practical guidance from Splunk experts on key ...

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...