Getting Data In

Best/optimum log file size?

lguinn2
Legend

I saw a recommendation that Splunk works better with smaller log files. But what does "small" mean?

I would assume that 2GB is small, but maybe not? I can set log file rotation based on size or time, and I wanted to use size to optimize Splunk's ability to read the files...

I found a couple of other answers that touch on this subject, including one that says "there isn't really a max size except for file system limits." Okay for the maximum, but what is optimum? No one actually says "it's better if your log files don't exceed size X."

muebel
SplunkTrust
SplunkTrust

My take on this is that it might not be that Splunk work's better per se, but that it is better to setup smaller files through rotation by the fact that, since Splunk is indexing the contents of the files, you don't have to leave as much local on the system's disk.

0 Karma

masonmorales
Influencer

1 GB is the recommended size that I heard at conf 2015.

lguinn2
Legend

Can you give more info on where you heard this? Just curious...

muebel
SplunkTrust
SplunkTrust

Was that from the "Logging: Proven operational practices" section? I couldn't find a direct match for "Splunk works better with smaller log files"

0 Karma

lguinn2
Legend

Yes, the direct quote is "Use smaller log files."

0 Karma