Could you please help me to fix this out.
I am trying to monitor a large folder containing multiple files on the Splunk server itself.
It says I can monitor a folder but when I go to Data Input to add a new input to monitor this folder it only allows me to select the file in the folder. I can't select the folder as a whole to be monitored.
Are the files all of the same type so they'd have the same sourcetype? If they are all going to be the same sourcetype you can hit the 'Skip' button on the data previewer app and specify the directory, and then check the 'More Settings' box to do the appropriate settings. Going through the data preview app on one file will take you to the same place and you can modify to have the whole directory monitored there as well.
the instructions for doing this are here:
is it possible that you're choosing "upload a file" instead of "Continuously index data from a file or directory this Splunk instance can access" ?
are you using 'data preview'? i think data preview works only on a single file (which makes sense, since it's previewing handling of a particular source type). try skipping the preview option and going directly to the 'add new' page.
as i suggest below, is it possible that you're choosing "upload a file" instead of "Continuously index data from a file or directory this Splunk instance can access" ?
This is what I did.
1.Manager > Data Inputs > Files & Directoris > Add New
2.Upload and Index a file
3.When I browse it only allows me to select the file not the full folder containing all the files I wanted. Can you please try creating few sample files and put in it a folder and try uploading?
Option it self says "Upload and Index a File" So i get the feeling that you can't upload a folder ? wouldn't it?
But if you see under Manager >Data Inputs and read whats written under Files and Directoris is says "Upload a file, index a local file, or monitor an entire directory. "
indikaw, After clicking Add New, you'll be at the Data Preview screen. Instead of clicking "Browse Server", click on "Skip Preview", and then "Continue". At the next screen, specify the full path to the directory you want to monitor and click Save.
After saving how can I ensure that it will load and index all the data including sub folders/files ?
How do I confirm all the data has been loaded and indexed successfully?
Which method can I use to check the progress of the indexing?
Since there is 200Gb of data to be indexed I assume it will take about couple of days? I am running this Splunk on Windows and files are located in the same same Splunk server