Hi our logs stop forwarding for a while after they have been archived, and this causes us to miss out on valuable data, how can I make splunk start monitoring that log after the old one has been renamed and archived ?
Please share the inputs.conf stanza for the files in question.
where can I find that file?
It's on the forwarder. Or you can run btool on the forwarder.
splunk btool inputs list | grep -v "system\/default"
Just copy and paste the stanza for the rotated files.
hi
I don't want the rotated log because its being archived, I want the new log that's been generated.. because somehow after log rotation logs stop being forwarded.
That is understood. The solution is to correct your inputs.conf file, but we have to see the current setting to know what needs correcting.
[default]
index = default
_rcvbuf = 1572864
host = $decideOnStartup
[blacklist:$SPLUNK_HOME/etc/auth]
[blacklist:$SPLUNK_HOME/etc/passwd]
[monitor://$SPLUNK_HOME/var/log/splunk]
index = _internal
[monitor://$SPLUNK_HOME/var/log/watchdog/watchdog.log*]
index = _internal
[monitor://$SPLUNK_HOME/var/log/splunk/license_usage_summary.log]
index = _telemetry
[monitor://$SPLUNK_HOME/var/log/splunk/splunk_instrumentation_cloud.log*]
index = _telemetry
:
Which stanza is causing the problem? IOW, what is the name of the rotated file?
I suspect the [monitor://$SPLUNK_HOME/var/log/watchdog/watchdog.log*] or [monitor://$SPLUNK_HOME/var/log/splunk/splunk_instrumentation_cloud.log*] stanza is the cause and the rotated files have an additional extension after ".log" (like ".log.gz", for instance). If so, the solution is to add a blacklist attribute to the stanza(s) so files with the rotated extension are ignored.
are you able to tell me what do these lines do [monitor://$SPLUNK_HOME/var/log/watchdog/watchdog.log*] or [monitor://$SPLUNK_HOME/var/log/splunk/splunk_instrumentation_cloud.log*]
because watchdog.log and cloud.log are not the logs we are monitoring
The first of those stanzas monitors files /opt/log/var/log/watchdog directory with names beginning with "watchdog.log". The second of the stanzas monitors files /opt/log/var/log/splunk directory with names beginning with "splunk_instrumentation_cloud.log".