Getting Data In

Splunk Light Forwarder - Maximum file size for a monitored file?

g3s1oa
Explorer

Hi,

We have a splunk light forwarder monitoring a file that grows in excess of 5GB a day before rolling over. The roll over threshold cannot be changed. Splunk is having issues indexing this file and often time misses data, completely stops indexing data for this source (although other sources on the box seem fine) or indexes data on a different date than when the log was written. (for instance, an event will be written to the log on the 17th, but it will show up on our search head as the 14th or 15th). Unfortunately, there is only a time in the log and not a date.

Does anyone know if there is a maximum size a log can be for Splunk to still successfully monitor and forward that log? Is there an easy work around to the issue of the date not being present in the log?

Thanks!

delink
Communicator

With the date not present in the log, your best option is to have the filename being read have the date encoded in it somewhere.

Check out http://docs.splunk.com/Documentation/Splunk/6.0/Data/HowSplunkextractstimestamps for more details.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...