We have a system that is being monitored fine, every 2 weeks the users switch the logs to .log.1 etc etc and then start a new .log
For some reason splunk will not monitor the new .log file until the forwarder is restarted.
here is the inputs
[monitor:///server/abc-timings*.log]
disabled=false
index=prod_collect
sourcetype=prod_abc_timing_log
use this monitor stanza instead, only add a wildcard *
at the end
[monitor:///server/abc-timings*.log*]
disabled=false
index=prod_collect
sourcetype=prod_abc_timing_log
read here more:
https://docs.splunk.com/Documentation/Splunk/7.2.5/Data/Specifyinputpathswithwildcards
hope it helps
Hello, thanks for this, however it seems that this would then continue to read the .log1 etc that has been archived. The issue is that it doesnt read the newly created .log file until we restart the forwarder?
it will read any new created file, examples: .log1
OR .log2
OR .logblahblah
please read the link carefully and apply the relevant wildcard
Still: how does that solve his problem that after rolling .log to .log1 and creating a new .log file, splunk does not pick up (quickly) on that new .log file?
This is what we've found, all that it does is then try to re-index the file that is now .log1 as well as the .log
The issue is still that it does not index the newly created .log until we start the forwarder again.
once a file is changed to .log1 we arent interested in it anymore.
thank you @FrankVl! looks like i didnt fully understand the question.
@lavster did you try and use the crcSalt
and initCrcLength
?
read here:
https://docs.splunk.com/Documentation/Splunk/7.2.5/Admin/Inputsconf
Reading the initial documentation splunk doesnt advise using crcsalt for LogSwaps. So we havent gone down that route yet. we were just hoping someone else may have seen this issue previously.
"Do not use crcSalt = with rolling log files, or any other scenario in which logfiles get renamed or moved to another monitored location. Doing so prevents Splunk Enterprise from recognizing log files across the roll or rename, which results in the data being reindexed."
First you should check the _internal logging for that host at the time when the logs switches
index=_internal host=yourhost sourcetype=splunkd WatchedFile
You might find a file to small too check message there, if so the link below could help you.
http://docs.splunk.com/Documentation/Splunk/latest/Data/Howlogfilerotationishandled