topic Re: Monitor directory and files in Getting Data In

Monitor directory and files

azduke — Fri, 17 May 2019 23:11:38 GMT

Hello all, I am facing some problems and would love to get some help. I need to monitor a directory for file creation, the files are named something like 27074.log or 57321.log, it basically is a process number name with the .log extention, once the file is created I need to monitor the content of the file loogking for an error, and if the error is there I need to raise an alert, is this possible?

Re: Monitor directory and files

koshyk — Sat, 18 May 2019 07:34:51 GMT

Yes, this is basic function of Splunk

So best practice is
1. Install Splunk Universal forwarder in your client (or HF in some cases)
2. Create an app MY_LOG_INPUTS and create local/inputs.conf
3. In the inputs.conf , put a stanza [monitor:///absolute/location/of/*.log], put index where it should go to, and sourcetype

Splunk will then take care automatically. Please read inputs.conf spec and outputs.conf spec to setup forwarding of logs to indexer

Re: Monitor directory and files

DavidHourani — Sat, 18 May 2019 09:35:38 GMT

Hi @azduke,

Seems like you're new to Splunk, welcome. Yes this is indeed possible. You can find a tutorial on how to monitor files in Splunk docs here :
https://docs.splunk.com/Documentation/Splunk/7.2.6/Data/Monitorfilesanddirectorieswithinputs.conf

This link is a bit technical as it will explain how to use the inputs.conf file. If you're looking for an easy solution go for this one as it's all GUI based and you can setup the data collection from a file without having to run any CLI configs :
https://docs.splunk.com/Documentation/Splunk/7.2.6/Data/Monitordata

In either way if the data you are looking to collect is not on the Splunk server itself then you will need to forward it to your splunk deployment first. You can use forwarder for that, some info here :
https://docs.splunk.com/Documentation/Splunk/7.2.6/Forwarding/Aboutforwardingandreceivingdata

Cheers,
David

Re: Monitor directory and files

azduke — Sat, 18 May 2019 18:34:00 GMT

Thank you very much David and Hoshyk, yes I am 100% new, I started a couple of weeks ago and I did not know it could be a directory, I thought it had to be a named file. I have a couple of weeks using Splunk and I was even thinking about first monitor the directory and calling a script when a file was created to move the file renaming it to somewhere else and there consume the file... Rookie overcomplication of things, I truly thank you both.

Re: Monitor directory and files

DavidHourani — Mon, 20 May 2019 11:10:46 GMT

most welcome man 😉 just let us know if you need anything 🙂

Re: Monitor directory and files

mathiasy123 — Mon, 06 Jul 2020 09:59:20 GMT

@DavidHourani

Hello, I did what you suggested about the monitoring directory in web GUI, but I had some warning:

"Data preview will be skipped, it is not supported for directories."

I have 2 files log in D:\Semester 7\Magang\Proyek\Proyek FDS that I want to monitor

Thanks in advanced