** Urgent ** - Question on how to specify DIRECTORY on unix system to index data from the files within directory

[mehal]

Hello All,

I need a help in indexing whole DIRECTORY to index data from files residing in directory.
My directory is " /proj_data " and there are some 250-300 files in this directory.

Here's what i do:

I add data from " a file or directory of files". I browse server to select directory. When i select directory and try to proceed further by clicking "select" button it remains disabled. But if i click a file withing /proj_data it gets enabled.

So i am not able to select whole directory but a single file only.

Any idea on how do i index chunk of files together ? or it is not possible with splunk ??

[MHibbin]

Hi,

It is possible, I think if you are using the web UI then you are also probably trying to preview the file before indexing... this, for fairly obvious reasons, will not work as the files in the directory may be completely different to each other. There should be an option to skip preview, and you also want to manually type in the path to the file.

You should read the following documentation (instead of re-inventing the wheel)...

http://docs.splunk.com/Documentation/Splunk/5.0/Data/Monitorfilesanddirectories

http://docs.splunk.com/Documentation/Splunk/5.0/Data/UseSplunkWeb

I would recommend setting up this monitor through the inputs.conf file, as this should give you more flexibility...

http://docs.splunk.com/Documentation/Splunk/5.0/Data/Editinputs.conf

Hope this helps,

MHibbin

[MHibbin]

from the search view (flashtimeline) you can search with index=_internal and search for some of yours files as search arguments (probably using wildcards to make things easier)

[MHibbin]

Have you made certain that Splunk has permission to read the files? i.e. The files either have global level read permissions, or the user running splunk is in a group that has permissions to read the file?

You should typically edit inputs.conf files in the local directory (e.g. $SPLUNK_HOME/etc/apps/search/local/inputs.conf), as the default folder is usually used for global defaults (i.e. if an app is posted on SB, it will have configs in default, allowing local users to customise local configs.)

[mehal]

Also could state if i specify data from splunkweb then those details appear in /local/inputs.conf or /default/inputs.conf?. Because i dont see any details when i added data input from splunkweb in inputs.conf in either location.

[mehal]

HI MHibbin,

I tried above approach but no luck. when i try with splunk web by selecting skip preview -> continuously index data...splunk can access -> full path (/proj_data/)-> save.
It does adds data input with equal number of files on data inputs. But i dont see any data getting indexed ? 😞

I also tried same with changing inputs.conf at /local/inputs.conf but same thing happens and no data getting indexed.

How can i verify that data is getting index.

My files are .csv files in /proj_data folder.