Our logfiles are named in the format Log.Activity.prod.###.txt where ### is random number. Also we want to leave out previous days log which would be in the format Log.Activity.prod.###.yyyy-mm-dd.txt (using the blacklist -).
We have setup splunk light forwarders and following is what we have on our Inputs.conf file:
[monitor://d:\LogFiles\prod\Log.Activity.prod.*] blacklist = - disabled = false sourcetype = Prod [monitor://d:\LogFiles\beta\Log.Activity.beta.*] blacklist = - disabled = false sourcetype = Beta [monitor://d:\LogFiles\alpha\Log.Activity.alpha.*] blacklist = - disabled = false sourcetype = Alpha
But for some reason splunk does not identify the file that is being logged to.
You might have more success with something like this:
[monitor://d:\LogFiles\prod] whitelist = Log.Activity.[a-zA-Z]+.[0-9]+.txt disabled = false sourcetype = Prod recursive = false
The whitelist avoids everything but your "current" logfile. (I'm not sure how wildcards in the monitor stanza and whitelist/blacklist interact -- something in the back of my mind says they don't get along, as Splunk internally might be using whitelist/blacklist to implement your wildcards.)
Strictly speaking, you aren't required to not monitor the 'older' files. As long as the first 256 bytes are the same, Splunk should recognize it as a rotated file and not re-index it even if the name changes.
You can turn on sourcetype auto classification - see http://www.splunk.com/base/Documentation/latest/Admin/Aboutdefaultfields . But, if you are going to manually specify the sourcetype in an inputs.conf stanza, it can only take on one value per stanza.
Note - I only updated one of your inputs.conf stanzas - you should be able to make up the other two based upon it.
I have not had luck with the above. But I do see the following error on the splunkd.log
10-27-2010 12:53:49.895 INFO TailingProcessor - Could not send data to output queue (parsingQueue), retrying...
10-27-2010 12:53:49.895 INFO TailingProcessor - ...continuing.
I even tried adding "crcsalt =
If you are having parsingQueue full type problems, some of these related answers may help:
There weren't any issues with my Inputs.conf, but after changing the license on splunk to use the forwarder license, the access to Inputs.conf file was removed for the profile splunk was running under. Resetting the permission on the splunk folder resolved the issue. Thanks.