Getting Data In

What happens if a large log file being monitored hasn't fully been forwarded at the time of rotation?

pkeller
Contributor

If I'm monitoring a very large logfile

[monitor:///home/me/logs]
whitelist = (myApp)\.log$

/home/me/logs/myApp.log

And at some point, a process rotates the file to:

/home/me/logs/OLD/myApp.log

If the file hasn't fully been forwarded at the time of rotation ... will:

  1. myApp.log be monitored in the new directory (assumed because OLD would be in scope for the monitored path)
  2. myApp.log be monitored in its entirety, or will Splunk still know the offset that was last indexed

Thank you.

If

0 Karma
1 Solution

hortonew
Builder

Splunk keeps track of the offset via the fishbucket. Even if the file is moved, it should only index what it hasn't already indexed. So moving it to a different directory shouldn't be a problem.

View solution in original post

0 Karma

hortonew
Builder

Splunk keeps track of the offset via the fishbucket. Even if the file is moved, it should only index what it hasn't already indexed. So moving it to a different directory shouldn't be a problem.

0 Karma
Get Updates on the Splunk Community!

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...