Splunk stops indexing log file when the log is switched to .log1

[lavster]

We have a system that is being monitored fine, every 2 weeks the users switch the logs to .log.1 etc etc and then start a new .log

For some reason splunk will not monitor the new .log file until the forwarder is restarted.

here is the inputs

[monitor:///server/abc-timings*.log]
disabled=false
index=prod_collect
sourcetype=prod_abc_timing_log

[adonio]

use this monitor stanza instead, only add a wildcard * at the end

[monitor:///server/abc-timings*.log*]
disabled=false
index=prod_collect
sourcetype=prod_abc_timing_log

read here more:

https://docs.splunk.com/Documentation/Splunk/7.2.5/Data/Specifyinputpathswithwildcards

hope it helps

[lavster]

Hello, thanks for this, however it seems that this would then continue to read the .log1 etc that has been archived. The issue is that it doesnt read the newly created .log file until we restart the forwarder?

[adonio]

it will read any new created file, examples: .log1 OR .log2 OR .logblahblah
please read the link carefully and apply the relevant wildcard

[FrankVl]

Still: how does that solve his problem that after rolling .log to .log1 and creating a new .log file, splunk does not pick up (quickly) on that new .log file?

[lavster]

This is what we've found, all that it does is then try to re-index the file that is now .log1 as well as the .log

The issue is still that it does not index the newly created .log until we start the forwarder again.

once a file is changed to .log1 we arent interested in it anymore.

[adonio]

thank you @FrankVl! looks like i didnt fully understand the question.
@lavster did you try and use the crcSalt and initCrcLength?
read here:
https://docs.splunk.com/Documentation/Splunk/7.2.5/Admin/Inputsconf

[lavster]

Reading the initial documentation splunk doesnt advise using crcsalt for LogSwaps. So we havent gone down that route yet. we were just hoping someone else may have seen this issue previously.

"Do not use crcSalt = with rolling log files, or any other scenario in which logfiles get renamed or moved to another monitored location. Doing so prevents Splunk Enterprise from recognizing log files across the roll or rename, which results in the data being reindexed."

[MattibergB]

First you should check the _internal logging for that host at the time when the logs switches
index=_internal host=yourhost sourcetype=splunkd WatchedFile

You might find a file to small too check message there, if so the link below could help you.
http://docs.splunk.com/Documentation/Splunk/latest/Data/Howlogfilerotationishandled