Getting Data In

Splunk Light Forwarder - Maximum file size for a monitored file?



We have a splunk light forwarder monitoring a file that grows in excess of 5GB a day before rolling over. The roll over threshold cannot be changed. Splunk is having issues indexing this file and often time misses data, completely stops indexing data for this source (although other sources on the box seem fine) or indexes data on a different date than when the log was written. (for instance, an event will be written to the log on the 17th, but it will show up on our search head as the 14th or 15th). Unfortunately, there is only a time in the log and not a date.

Does anyone know if there is a maximum size a log can be for Splunk to still successfully monitor and forward that log? Is there an easy work around to the issue of the date not being present in the log?



With the date not present in the log, your best option is to have the filename being read have the date encoded in it somewhere.

Check out for more details.

0 Karma
Get Updates on the Splunk Community!

Unify Your SecOps with Splunk Mission Control

In today’s post, I'm excited to share some recent Splunk Mission Control innovations. With Splunk Mission ...

Data Preparation Made Easy: SPL2 for Edge Processor

By now, you may have heard the exciting news that Edge Processor, the easy-to-use Splunk data preparation tool ...

Introducing Edge Processor: Next Gen Data Transformation

We get it - not only can it take a lot of time, money and resources to get data into Splunk, but it also takes ...