Archive

Splunk stops indexing log file when the log is switched to .log1

lavster
New Member

We have a system that is being monitored fine, every 2 weeks the users switch the logs to .log.1 etc etc and then start a new .log

For some reason splunk will not monitor the new .log file until the forwarder is restarted.

here is the inputs

[monitor:///server/abc-timings*.log]
disabled=false
index=prod_collect
sourcetype=prod_abc_timing_log

Tags (1)
0 Karma

adonio
SplunkTrust
SplunkTrust

use this monitor stanza instead, only add a wildcard * at the end

[monitor:///server/abc-timings*.log*]
disabled=false
index=prod_collect
sourcetype=prod_abc_timing_log

read here more:

https://docs.splunk.com/Documentation/Splunk/7.2.5/Data/Specifyinputpathswithwildcards

hope it helps

0 Karma

lavster
New Member

Hello, thanks for this, however it seems that this would then continue to read the .log1 etc that has been archived. The issue is that it doesnt read the newly created .log file until we restart the forwarder?

0 Karma

adonio
SplunkTrust
SplunkTrust

it will read any new created file, examples: .log1 OR .log2 OR .logblahblah
please read the link carefully and apply the relevant wildcard

0 Karma

FrankVl
Ultra Champion

Still: how does that solve his problem that after rolling .log to .log1 and creating a new .log file, splunk does not pick up (quickly) on that new .log file?

0 Karma

lavster
New Member

This is what we've found, all that it does is then try to re-index the file that is now .log1 as well as the .log

The issue is still that it does not index the newly created .log until we start the forwarder again.

once a file is changed to .log1 we arent interested in it anymore.

0 Karma

adonio
SplunkTrust
SplunkTrust

thank you @FrankVl! looks like i didnt fully understand the question.
@lavster did you try and use the crcSalt and initCrcLength?
read here:
https://docs.splunk.com/Documentation/Splunk/7.2.5/Admin/Inputsconf

0 Karma

lavster
New Member

Reading the initial documentation splunk doesnt advise using crcsalt for LogSwaps. So we havent gone down that route yet. we were just hoping someone else may have seen this issue previously.

"Do not use crcSalt = with rolling log files, or any other scenario in which logfiles get renamed or moved to another monitored location. Doing so prevents Splunk Enterprise from recognizing log files across the roll or rename, which results in the data being reindexed."

0 Karma

MattibergB
Path Finder

First you should check the _internal logging for that host at the time when the logs switches
index=_internal host=yourhost sourcetype=splunkd WatchedFile

You might find a file to small too check message there, if so the link below could help you.
http://docs.splunk.com/Documentation/Splunk/latest/Data/Howlogfilerotationishandled

0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!