topic Recursive monitoring of directories in Getting Data In

Recursive monitoring of directories

sushma7 — Thu, 01 May 2014 12:19:52 GMT

Hi Team,

I have a folder by name Mumbai under C drive with subfolders in it.

If i edit the inputs.conf file as monitor://C:\Splunk\NPCI\Mumbai\* , will that monitor all recursively all the sub folders?

Correct me if my command is wrong.

Thanks & Regards,
Sushma.

Re: Recursive monitoring of directories

sushma7 — Thu, 01 May 2014 12:22:26 GMT

the command is not showing up properly, it is monitor://C:\Splunk\NPCI\Mumbai*

Re: Recursive monitoring of directories

sushma7 — Thu, 01 May 2014 12:23:18 GMT

There are forward splashes between Splunk, NPCI and Mumbai, I dont knew as why it is not showing up

Re: Recursive monitoring of directories

linu1988 — Thu, 01 May 2014 12:26:13 GMT

Hello Sushma,
It will look for the current directory. You need to set the recursive option to True in the stanza.

recursive = [true|false]
* If false, Splunk will not monitor subdirectories found within a
monitored directory.
* Defaults to true.

Your Monitor stanza

[monitor://C:\Splunk\NPCI\Mumbai]
recursive = true

Thanks

Re: Recursive monitoring of directories

sushma7 — Thu, 01 May 2014 12:29:41 GMT

So my command needs to be monitor://C:\Splunk\NPC\Munbai*
recursive = true

Is that right?

Re: Recursive monitoring of directories

linu1988 — Thu, 01 May 2014 12:37:23 GMT

yes that should be fine if you are looking for reading out all the files recursively. May be you want to put sourcetype as well.

Re: Recursive monitoring of directories

sushma7 — Thu, 01 May 2014 13:14:17 GMT

Should there be any slash after * in the above command?

Re: Recursive monitoring of directories

somesoni2 — Thu, 01 May 2014 13:46:03 GMT

Go to link below and search for "Note concerning wildcards and monitor" in the page

http://docs.splunk.com/Documentation/Splunk/latest/admin/Inputsconf

Re: Recursive monitoring of directories

sushma7 — Fri, 02 May 2014 08:54:52 GMT

As directed by you, I have used the above syntax i.e. monitor://C:\Splunk\NPCI\Mumbai*
recursive = true

But it is not getting indexed automatically, Is there anything else that I need to include?

Note: The folder Mumabi has 3 subfolders by names Axis, Hyderabad and NPCI. I need to monitor all these folders and contents within them.

Kindly correct the command if I am wrong somewhere.

Re: Recursive monitoring of directories

linu1988 — Sat, 03 May 2014 15:03:02 GMT

splunk needs to recognize the files. What are the file format?

Re: Recursive monitoring of directories

splunker12er — Sat, 03 May 2014 16:25:02 GMT

Is the monitored folder have active logs ?
Try some editing in the files under the monitored path.. you should get data at search head

Re: Recursive monitoring of directories

sushma7 — Sun, 04 May 2014 15:29:24 GMT

The files within the folder are text document. What is meant by active logs? What should I change?

Re: Recursive monitoring of directories

linu1988 — Sun, 04 May 2014 16:29:48 GMT

did you change the monitor stanza which i have specified? If the files are not monitored until now, it will be monitored.

Re: Recursive monitoring of directories

sushma6 — Sun, 04 May 2014 16:34:16 GMT

Yes I did the same as you specified i.e. [monitor://C:\Splunk\NPCI\Mumbai]
recursive = true
index = mdc
But it's not detecting, I even want all of them to be indexed into the mdc that I have created rather than they getting indexed into default main. Hence I included index = mdc

Re: Recursive monitoring of directories

splunker12er — Sun, 04 May 2014 16:47:45 GMT

whatever may be the file format , splunk will monitor the folder that you mentioned in the monitor stanza.
Create a inputs.conf under the path:
/etc/apps/search/local/

[monitor://C:\Splunk\NPCI\Mumbai]
recursive = true
index = mdc
sourcetype = Mumbai

test:
Copy a text file inside the folder that you want to monitor and paste the file inside the same folder and rename the file. or edit the same and copy paste some contents for recent logs. Also check your outputs.conf is correct

Re: Recursive monitoring of directories

sushma6 — Sun, 04 May 2014 17:05:19 GMT

Under search folder I could not find any local folder, there's only 3 folders under it by names default, lookups and metadata. I think if we edit inputs.conf under /etc/system/local its enough, is it not so?

Re: Recursive monitoring of directories

splunker12er — Sun, 04 May 2014 17:11:33 GMT

that should be fine.. just give a try .. if folders doesn't exit create folders under the path i mentioned

Re: Recursive monitoring of directories

linu1988 — Sun, 04 May 2014 17:16:12 GMT

It must be permission issue. Try giving proper access or move the files to another drive like E: or F: , it will start reading everything...

[monitor://E:\Splunk\NPCI\Mumbai]
recursive = true
index=mdc

Re: Recursive monitoring of directories

sushma6 — Mon, 28 Sep 2020 16:31:45 GMT

I tried this:

[monitor://C:\Splunk\NPCI\Mumbai]
recursive = true
index = mdc
sourcetype = Mumbai

But not working.

It is not permission issue, because when I read individual files it is getting indexed i.e

[monitor://C:\Splunk\NPCI\Mumbai\2014_05_02\App01]
index = mdc

[monitor://C:\Splunk\NPCI\Mumbai\2014_05_03\App01]
index = mdc

but when I give the complete folder i.e.

[monitor://C:\Splunk\NPCI\Mumbai]
index = mdc

to monitor its not getting indexed.

Re: Recursive monitoring of directories

splunker12er — Sun, 04 May 2014 18:10:52 GMT

Considering all the files are text files,

[monitor://C:\Splunk\NPCI\Mumbai]
recursive = true
index = mdc
sourcetype = Mumbai
whitelist = \.txt$

1.Delete the early index data in search head (index=mdc|delete)
2.Edit your inputs.conf as above
3.Restart the forwarder