topic Re: Monitor by File Name Only in Getting Data In

Monitor by File Name Only

gallantalex — Fri, 08 Oct 2010 02:50:24 GMT

Hi, I would like to monitor all the web.config files on my machine and then forward the results to a Splunk receiver. I have tried creating a file monitoring data input of:

[monitor://.../web.config]
disabled = false
sourcetype = webconfig

[monitor://...]
disabled = false
sourcetype = webconfig
whitelist = web.config

But nothing seems to work. Its probably unlikely that something like this would work because of the overhead of searching in all directories, but I was hoping Splunk would be able to. Maybe search all directories once a day and memorize the location of the other web.config files. Thanks.

Re: Monitor by File Name Only

southeringtonp — Fri, 08 Oct 2010 06:30:01 GMT

As you say, searching across all directories involves a lot of overhead. Even if it would work, it's best avoided.

Also, note that when you start with ..., you are not really providing a root for the search to begin. Normally a monitor stanza is going to start with a drive letter (I'm assuming Windows based on your filename). I doubt very much that Splunk will enumerate all filesystems just for the sake of fulfilling a wildcard.

Your best bet would be a happy middle ground. Try to determine the directories where web.config is likely to be located, and then create multiple inputs accordingly.

If you are not already aware, remember that you can use splunk list monitor from the command line to see which files Splunk has found to monitor.

Re: Monitor by File Name Only

gallantalex — Fri, 08 Oct 2010 06:48:36 GMT

Thanks, I figured that was the case. Also the splunk list monitor command was very helpful.

Re: Monitor by File Name Only

Lowell — Fri, 08 Oct 2010 21:18:19 GMT

Sounds like you are looking for the crawl command. You can use this as part of a scheduled search (run it daily to monthly, depending on how often new files could show up). The crawl command will scan your directory and look for potential log files.

Something like this could do the trick:

| crawl root="/var/log;/opt/log" | search isfile=True source="*/web.config" NOT status="added" | input add

NOTE: You should take the "| input add" off the end until your confident that the search is going to do what you want.

You may need to update crawl.conf, since it looks that bad_extensions_list includes "config" which would prevent this file from showing up.

Re: Monitor by File Name Only

athorat — Thu, 13 Aug 2015 00:23:03 GMT

Hi We are looking to get similar info for all .csv files in a specific folder and not the actual content of the files.
ver 6.1.3
I did not quite get the changes using fschange or the query above....
Thanks for looking into this.