Getting Data In

What happens if a large log file being monitored hasn't fully been forwarded at the time of rotation?

pkeller
Contributor

If I'm monitoring a very large logfile

[monitor:///home/me/logs]
whitelist = (myApp)\.log$

/home/me/logs/myApp.log

And at some point, a process rotates the file to:

/home/me/logs/OLD/myApp.log

If the file hasn't fully been forwarded at the time of rotation ... will:

  1. myApp.log be monitored in the new directory (assumed because OLD would be in scope for the monitored path)
  2. myApp.log be monitored in its entirety, or will Splunk still know the offset that was last indexed

Thank you.

If

0 Karma
1 Solution

hortonew
Builder

Splunk keeps track of the offset via the fishbucket. Even if the file is moved, it should only index what it hasn't already indexed. So moving it to a different directory shouldn't be a problem.

View solution in original post

0 Karma

hortonew
Builder

Splunk keeps track of the offset via the fishbucket. Even if the file is moved, it should only index what it hasn't already indexed. So moving it to a different directory shouldn't be a problem.

0 Karma
Get Updates on the Splunk Community!

Splunk at Cisco Live 2025: Learning, Innovation, and a Little Bit of Mr. Brightside

Pack your bags (and maybe your dancing shoes)—Cisco Live is heading to San Diego, June 8–12, 2025, and Splunk ...

Splunk App Dev Community Updates – What’s New and What’s Next

Welcome to your go-to roundup of everything happening in the Splunk App Dev Community! Whether you're building ...

The Latest Cisco Integrations With Splunk Platform!

Join us for an exciting tech talk where we’ll explore the latest integrations in Cisco + Splunk! We’ve ...