Directory monitoring not picking up new files

[nate1]

I have a folder I setup using the files/directories which I gave the directory name (F:\test). It indexed the single file I had in there. I have since then placed another file (of the same sourcetype) into the directory, yet the number of entries hasn't increased when I look at the search screen. It's been a while (>15 min). Any ideas? Thanks!

[the_wolverine]

Perhaps you copied the exact same file that was already indexed? In that case, a special configuration setting needs to be added in order for Splunk to index the new file because, based on the content of that copied (duplicated) file, it has already been indexed.

The solution in this case would be to add the following config setting to your inputs.conf (there is no way to set this from UI, you'll have to manually edit the inputs.conf file):

crcSalt = <SOURCE>

[the_wolverine]

Yes, this is a great point, Lowell. In this case the assumption is also that this is entirely a test configuration that will be dropped once the test is complete. For testing purposes we do not like to ask the Splunk user to modify their file. They should be modifying Splunk to work with their existing files.

As Lowell mentions, crcSalt should not be used in all situations but would certainly be the answer to the situation where you have multiple files of the same size and content.

[Lowell]

If you did copy a log file, because you are setting up a new splunk instance or testing a new sourcetype or something like that. That I would suggest simply modifying something trivial in the first event of your log file. (A single character change will do the trick) Using the crcSalt option will work, but it also has some other undesirable long-term side-effects and can be avoided in many cases. For example, by default splunk recognizes and ignores renamed (rotated) log files, but this setting disables that feature. Basically, I suggest keeping this as a last resort.

[Jeremiah]

Can you verify the data hasn't been indexed yet by searching for it? I've been able to find data in the index before the dashboard is updated. But if the data isn't even getting indexed, search the _internal index for any errors associated with reading the file. Some reasons why the file might not be indexed: file permissions, its getting typed as a binary, or it was previously indexed in Splunk.

[abhayneilam]

lets take another example :

I have two files which is of same type means :
file_one contains:
name|age|sex|location
xyz|45|M|kol
mno|50|F|mum

and file_two contains:
name|age|sex|location
abc|60|M|hyd
lkg|100|M|ker

these two files are in the same directory, but while importing the directory it is talking only the first file not the second file.I cant see the content of the second file while searching..please suggest