How do you monitor a folder in Splunk for a specific index or source type?

[maryamchar]

Hello,

I'm new to Splunk. I'm using the Search and Reporting app only. I want to upload data using monitor, however, my question is there a way to monitor a certain folder for a specific source type and Index? I want Splunk to keep monitoring that source type, and that index, all the time and all my files are located on the same folder.

How would i tell Splunk to monitor a certain source type or index ?

Please let me know, if you can show me an example that would be great, Thank you in advance!

[richgalloway · Accepted solution]

You can't monitor for specific indexes or sourcetypes. You can, however, monitor for specific data using regular expressions. Events that match those expressions can be indexed and everything else can be discarded. See "Route and filter data" at https://docs.splunk.com/Documentation/Splunk/7.1.2/Forwarding/Routeandfilterdatad.

[maryamchar]

Thank you! Can you monitor an entire folder instead that has a lot of files that keep coming to that folder ? using the monitor (add data) tool. ? If yes how should i approch that ? Thank you!

[richgalloway]

Yes, you can monitor a directory as well as individual files. Using the Add Data tool, select the Monitor option followed by "Files & Directories". The rest should be self-explanatory.

[maryamchar]

Thank you! But each of those files in that directory of folder, can't have it's own sourcetype name correct ?
I'm trying to find if there is a way to set each of those files to have it's own sourcetype name, and all of them have CSV as sourcetype as well. But as of now i can only set one sourcetype name but thats for all my files not a specific one.
Also, what does the blacklist and whitelist do in this case ? Again thank you for your respose i appreciate it!

[richgalloway]

Generally speaking, yes, each monitor stanza specifies a single sourcetype. There are ways to use transforms.conf to change that, though.
Whitelist and blacklist tell Splunk which files in the directory it should read or ignore, respectively.