I've got a Win 2008 Web server, and the layout on the disk is as follow:
C:\inetpub\sites\www.fqdn.com\logs\
C:\inetpub\sites\www.fqdn.com\www\
C:\inetpub\sites\another.fqdn.com\logs\
C:\inetpub\sites\another.fqdn.com\www\
etc..
I'm trying to set up a file monitor for C:\inetpub\sites*\logs\ so that only the log files are monitored (and later I want to put on fschange monitoring on the www files (html/php/aspx etc)). I've configured this in search app inputs.conf:
[monitor://C:\inetpub\sites\...\logs\]
sourcetype = iis
disabled = false
Somehow this adds C:\inetpub\sites\*\
to the monitor list, making splunk monitor the www files as well.
What have I missed in my config?
Creating a monitor for the directory C:\inetpub\sites
and filtering the files using a whitelist is probably the best option:
[monitor://C:\inetpub\sites\]
sourcetype = iis
disabled = false
whitelist=\\logs\\
Finally. This worked:
[monitor://C:\inetpub\sites\]
sourcetype = iis
followTail = 1
whitelist = \\logs\\*
disabled = 0
Note the last star in the whitelist (*)
Finally. This worked:
[monitor://C:\inetpub\sites\]
sourcetype = iis
followTail = 1
whitelist = \\logs\\*
disabled = 0
Note the last star in the whitelist (*)
I have subfolders under \logs.
C:\inetpub\sites\www.fqdn.com\logs\
C:\inetpub\sites\www.fqdn.com\logs\W3SVC4\ (the log files are here - *.log)
C:\inetpub\sites\www.fqdn.com\logs\FTPSVC4\ (log files are here - *.log)
C:\inetpub\sites\www.fqdn.com\www\
C:\inetpub\sites\www.fqdn.com\www\xxx\
C:\inetpub\sites\www.fqdn.com\www\yyy\
C:\inetpub\sites\another.fqdn.com\logs\
C:\inetpub\sites\another.fqdn.com\logs\W3SVC7\
C:\inetpub\sites\another.fqdn.com\logs\FTPSVC7\
C:\inetpub\sites\another.fqdn.com\www\
C:\inetpub\sites\another.fqdn.com\www\ (webfiles files here)
C:\inetpub\sites\another.fqdn.com\www\aaa\ (more webfiles here)
C:\inetpub\sites\another.fqdn.com\www\bbb\ (more webfiles here)
What I want, which I think you guys understand, is to monitor all log files under \logs\, no matter what the fqdn folder name is, and what folders are under \logs.
I confirmed that if I monitor the C:\inetpub\sites\lebara.stag.carrot.no\logs\W3SVC4\
the logs are indexed (as it should), but I can't seem to get the correct config with wildcards etc...
Update:
If i add whitelist=\\logs\\
I get this output from splunk list monitor
:
C:\inetpub\sites\
C:\inetpub\sites\www.fqdn.com
C:\inetpub\sites\www.fqdn.com\logs
C:\inetpub\sites\www.fqdn.com\logs\ProfileWS
C:\inetpub\sites\www.fqdn.com\logs\W3SVC3
C:\inetpub\sites\www.fqdn.com\www
C:\inetpub\sites\www.fqdn.com\www\App_Data
C:\inetpub\sites\www.fqdn.com\www\aspnet_client
etc... (for each website)
If I add a '*' after the last backslash - whitelist=\\logs\\*
- I get the logfiles in the monitor list at least, but still \www\
:
C:\inetpub\sites\
C:\inetpub\sites\www.fqdn.com
C:\inetpub\sites\www.fqdn.com\logs
C:\inetpub\sites\www.fqdn.com\logs\ProfileWS
C:\inetpub\sites\www.fqdn.com\logs\ProfileWS\ProfileWS.error.log
C:\inetpub\sites\www.fqdn.com\logs\ProfileWS\ProfileWS.log
C:\inetpub\sites\www.fqdn.com\logs\W3SVC3
C:\inetpub\sites\www.fqdn.com\logs\W3SVC3\u_ex100725.log
C:\inetpub\sites\www.fqdn.com\logs\W3SVC3\u_ex100726.log
C:\inetpub\sites\www.fqdn.com\logs\W3SVC3\u_ex100727.log
C:\inetpub\sites\www.fqdn.com\www
C:\inetpub\sites\www.fqdn.com\www\App_Data
C:\inetpub\sites\www.fqdn.com\www\aspnet_client
I'm getting quite frustrated here 😞 And regex is almost like greek to me.
Update 2010-07-29:
I'm now running with the whitelist = \\logs\\
config, but no logs gets sent to the splunk indexer. Checking splunkd.log I see this (and lots of the same kind):
07-29-2010 16:07:59.495 INFO TailingProcessor - No configurations match, will ignore path='C:\inetpub\sites\www.fqdn.com\www\bin\CIF.xml'.
07-29-2010 16:07:59.511 INFO TailingProcessor - No configurations match, will ignore path='C:\inetpub\sites\another.fqdn.com\logs\W3SVC7\u_ex100729.log'.
07-29-2010 16:29:20.013 INFO TailingProcessor - No configurations match, will ignore path='C:\inetpub\sites\www.fqdn.com\www\bin\CIF.xml'.
As you all can see, it doesn't match the \logs\ folder and therefor the logs aren't sent to my indexer... Isn't this strange?
Creating a monitor for the directory C:\inetpub\sites
and filtering the files using a whitelist is probably the best option:
[monitor://C:\inetpub\sites\]
sourcetype = iis
disabled = false
whitelist=\\logs\\
the whitelist doesn't work for my splunk lightforwarder. Even though the simple whitelist should be correct it is not forwarded.
By testing this simple regex in RegexBuddy (http://answers.splunk.com/questions/5092/regex-crash-course) with the entire folder structure I have I see whitelist=\\logs\\
is correct (not that I doubted). My question is then, why does splunk list monitor
list all the other folders?
I do presume that splunk list monitor
lists everything thats being monitored...
It did not work, still lists the \www\ files&folders as well. Do I need a blacklist as well then? Or should it only grab the \logs\ folder and nothing else?
I'm using 4.1.4 btw.
I will try. Will it work in combination with fschange later when I want to add that to the \www\ folder for tracking changes to the html/php/aspx/js/etc files? Would it be as simple as creating a fschange and a 'whitelist=\www\' to do this?