Getting Data In

What happens if a large log file being monitored hasn't fully been forwarded at the time of rotation?

pkeller
Contributor

If I'm monitoring a very large logfile

[monitor:///home/me/logs]
whitelist = (myApp)\.log$

/home/me/logs/myApp.log

And at some point, a process rotates the file to:

/home/me/logs/OLD/myApp.log

If the file hasn't fully been forwarded at the time of rotation ... will:

  1. myApp.log be monitored in the new directory (assumed because OLD would be in scope for the monitored path)
  2. myApp.log be monitored in its entirety, or will Splunk still know the offset that was last indexed

Thank you.

If

0 Karma
1 Solution

hortonew
Builder

Splunk keeps track of the offset via the fishbucket. Even if the file is moved, it should only index what it hasn't already indexed. So moving it to a different directory shouldn't be a problem.

View solution in original post

0 Karma

hortonew
Builder

Splunk keeps track of the offset via the fishbucket. Even if the file is moved, it should only index what it hasn't already indexed. So moving it to a different directory shouldn't be a problem.

0 Karma
Get Updates on the Splunk Community!

Splunk Observability for AI

Don’t miss out on an exciting Tech Talk on Splunk Observability for AI!Discover how Splunk’s agentic AI ...

Splunk Enterprise Security 8.x: The Essential Upgrade for Threat Detection, ...

Watch On Demand the Tech Talk on November 6 at 11AM PT, and empower your SOC to reach new heights! Duration: ...

Splunk Observability as Code: From Zero to Dashboard

For the details on what Self-Service Observability and Observability as Code is, we have some awesome content ...