Getting Data In
Highlighted

Monitor a folder

Explorer

Hi,

Could you please help me to fix this out.
I am trying to monitor a large folder containing multiple files on the Splunk server itself.
It says I can monitor a folder but when I go to Data Input to add a new input to monitor this folder it only allows me to select the file in the folder. I can't select the folder as a whole to be monitored.

Highlighted

Re: Monitor a folder

Splunk Employee
Splunk Employee

Are the files all of the same type so they'd have the same sourcetype? If they are all going to be the same sourcetype you can hit the 'Skip' button on the data previewer app and specify the directory, and then check the 'More Settings' box to do the appropriate settings. Going through the data preview app on one file will take you to the same place and you can modify to have the whole directory monitored there as well.

http://blogs.splunk.com/2010/02/11/sourcetypes-gone-wild/

Highlighted

Re: Monitor a folder

Splunk Employee
Splunk Employee

the instructions for doing this are here:
http://docs.splunk.com/Documentation/Splunk/5.0.1/Data/UseSplunkWeb

is it possible that you're choosing "upload a file" instead of "Continuously index data from a file or directory this Splunk instance can access" ?

are you using 'data preview'? i think data preview works only on a single file (which makes sense, since it's previewing handling of a particular source type). try skipping the preview option and going directly to the 'add new' page.

Highlighted

Re: Monitor a folder

Explorer

Yes i did skip and when I brows to the file it only allows me to select one file, where as I want to selct the main folder containing all the sub folders and files.

0 Karma
Highlighted

Re: Monitor a folder

Splunk Employee
Splunk Employee

as i suggest below, is it possible that you're choosing "upload a file" instead of "Continuously index data from a file or directory this Splunk instance can access" ?

0 Karma
Highlighted

Re: Monitor a folder

Explorer

Hi,

This is what I did.
1.Manager > Data Inputs > Files & Directoris > Add New
2.Upload and Index a file
3.When I browse it only allows me to select the file not the full folder containing all the files I wanted. Can you please try creating few sample files and put in it a folder and try uploading?

Option it self says "Upload and Index a File" So i get the feeling that you can't upload a folder ? wouldn't it?

But if you see under Manager >Data Inputs and read whats written under Files and Directoris is says "Upload a file, index a local file, or monitor an entire directory. "

0 Karma
Highlighted

Re: Monitor a folder

Splunk Employee
Splunk Employee

indikaw, After clicking Add New, you'll be at the Data Preview screen. Instead of clicking "Browse Server", click on "Skip Preview", and then "Continue". At the next screen, specify the full path to the directory you want to monitor and click Save.

0 Karma
Highlighted

Re: Monitor a folder

Explorer

Hi,
Question 1
After saving how can I ensure that it will load and index all the data including sub folders/files ?
Question 2
How do I confirm all the data has been loaded and indexed successfully?
Question 3
Which method can I use to check the progress of the indexing?

Question 4
Since there is 200Gb of data to be indexed I assume it will take about couple of days? I am running this Splunk on Windows and files are located in the same same Splunk server

Highlighted

Re: Monitor a folder

Explorer

Can someone please help me to get this thing right?> Appreciated.

0 Karma