Will monitored files that are renamed be reindexed?

[joshuapetitt]

I have an application that uses rolling logging. The rolling logging works as such:

• logs are initially written to "log_file.txt"
• after a certain time period, the "log_file.txt" is renamed to "log_file.txt.1", and an empty "log_file.txt" is created to start writing to.
• after a certain time period, the "log_file.txt.1" is renamed to "log_file.txt.2", "log_file.txt" is renamed to "log_file.txt.1" and an empty "log_file.txt" is created to start writing to.
• this process continues until X number of files are written, after that the oldest file file is deleted.

So if I monitor all of the files that match "log_file.*", will I have a bunch of duplicated data each time the file rolls over?

Should I just monitor "log_file.txt" in this instance?

[micahkemp · Accepted solution]

The answer is somewhat "it depends", but the default settings are likely sane enough to prevent re-indexing old data. Splunk uses the first initCrcLength bytes of a file to determine if it is a new file or not. That setting can be changed as necessary, and other settings can also be changed to force Splunk to use the filename to assist it in determining uniqueness, but it sounds like you want the normal behavior that a monitor input would give you.

For more information on the different settings you can use for your inputs check out the inputs.conf spec.

[MuS]

Did you mean inputs.conf http://docs.splunk.com/Documentation/Splunk/latest/Admin/Inputsconf ?
props.conf only has initCrcLength but anything else is configurable in inputs.conf

cheers, MuS

[micahkemp]

Yes, thanks! I had linked to the correct page but put props for the text instead.

[woodcock]

It also very much depends on whether you are using crcSalt = <SOURCE> or not.