Getting Data In

What happens if a large log file being monitored hasn't fully been forwarded at the time of rotation?

pkeller
Contributor

If I'm monitoring a very large logfile

[monitor:///home/me/logs]
whitelist = (myApp)\.log$

/home/me/logs/myApp.log

And at some point, a process rotates the file to:

/home/me/logs/OLD/myApp.log

If the file hasn't fully been forwarded at the time of rotation ... will:

  1. myApp.log be monitored in the new directory (assumed because OLD would be in scope for the monitored path)
  2. myApp.log be monitored in its entirety, or will Splunk still know the offset that was last indexed

Thank you.

If

0 Karma
1 Solution

hortonew
Builder

Splunk keeps track of the offset via the fishbucket. Even if the file is moved, it should only index what it hasn't already indexed. So moving it to a different directory shouldn't be a problem.

View solution in original post

0 Karma

hortonew
Builder

Splunk keeps track of the offset via the fishbucket. Even if the file is moved, it should only index what it hasn't already indexed. So moving it to a different directory shouldn't be a problem.

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) v3.54.0

The Splunk Threat Research Team (STRT) recently released Enterprise Security Content Update (ESCU) v3.54.0 and ...

Using Machine Learning for Hunting Security Threats

WATCH NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more for ...

New Learning Videos on Topics Most Requested by You! Plus This Month’s New Splunk ...

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...