Getting Data In

Monitor directory and files

azduke
Engager

Hello all, I am facing some problems and would love to get some help. I need to monitor a directory for file creation, the files are named something like 27074.log or 57321.log, it basically is a process number name with the .log extention, once the file is created I need to monitor the content of the file loogking for an error, and if the error is there I need to raise an alert, is this possible?

Tags (1)
0 Karma
1 Solution

DavidHourani
Super Champion

Hi @azduke,

Seems like you're new to Splunk, welcome. Yes this is indeed possible. You can find a tutorial on how to monitor files in Splunk docs here :
https://docs.splunk.com/Documentation/Splunk/7.2.6/Data/Monitorfilesanddirectorieswithinputs.conf

This link is a bit technical as it will explain how to use the inputs.conf file. If you're looking for an easy solution go for this one as it's all GUI based and you can setup the data collection from a file without having to run any CLI configs :
https://docs.splunk.com/Documentation/Splunk/7.2.6/Data/Monitordata

In either way if the data you are looking to collect is not on the Splunk server itself then you will need to forward it to your splunk deployment first. You can use forwarder for that, some info here :
https://docs.splunk.com/Documentation/Splunk/7.2.6/Forwarding/Aboutforwardingandreceivingdata

Cheers,
David

View solution in original post

0 Karma

azduke
Engager

Thank you very much David and Hoshyk, yes I am 100% new, I started a couple of weeks ago and I did not know it could be a directory, I thought it had to be a named file. I have a couple of weeks using Splunk and I was even thinking about first monitor the directory and calling a script when a file was created to move the file renaming it to somewhere else and there consume the file... Rookie overcomplication of things, I truly thank you both.

0 Karma

DavidHourani
Super Champion

most welcome man 😉 just let us know if you need anything 🙂

0 Karma

mathiasy123
Path Finder

@DavidHourani 

Hello, I did what you suggested about the monitoring directory in web GUI, but I had some warning: 

"Data preview will be skipped, it is not supported for directories."

 

I have 2 files log in D:\Semester 7\Magang\Proyek\Proyek FDS that I want to monitor

 

mathiasy123_0-1594029534679.png

 

 
 

Thanks in advanced

 

Tags (1)
0 Karma

DavidHourani
Super Champion

Hi @azduke,

Seems like you're new to Splunk, welcome. Yes this is indeed possible. You can find a tutorial on how to monitor files in Splunk docs here :
https://docs.splunk.com/Documentation/Splunk/7.2.6/Data/Monitorfilesanddirectorieswithinputs.conf

This link is a bit technical as it will explain how to use the inputs.conf file. If you're looking for an easy solution go for this one as it's all GUI based and you can setup the data collection from a file without having to run any CLI configs :
https://docs.splunk.com/Documentation/Splunk/7.2.6/Data/Monitordata

In either way if the data you are looking to collect is not on the Splunk server itself then you will need to forward it to your splunk deployment first. You can use forwarder for that, some info here :
https://docs.splunk.com/Documentation/Splunk/7.2.6/Forwarding/Aboutforwardingandreceivingdata

Cheers,
David

0 Karma

koshyk
Super Champion

Yes, this is basic function of Splunk

So best practice is
1. Install Splunk Universal forwarder in your client (or HF in some cases)
2. Create an app MY_LOG_INPUTS and create local/inputs.conf
3. In the inputs.conf , put a stanza [monitor:///absolute/location/of/*.log], put index where it should go to, and sourcetype

Splunk will then take care automatically. Please read inputs.conf spec and outputs.conf spec to setup forwarding of logs to indexer

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...