After setting up Splunk to monitor a folder, why is only the first log file getting indexed?

[instigardo]

Hi

I have set up Splunk to monitor a particular folder for logs, but somehow it picks only the 1st log file added to the folder, not the latter ones. Can you help solving this issue please?
The logs are shell created logs having filename *.log

I changed the name of the file it indexed and that file is getting indexed fine.

[lguinn2]

How long are you waiting before you decide that Splunk is only indexing the first file in the folder? If the first file in the folder is very active or very large, it may take Splunk some time to completely index that file and move on to the next file.

Also, as @ppablo said, you have not given the community much information to work with. I am just guessing here. It would be helpful to see: (1) the inputs.conf monitor stanza (2) the list of the files in the directory with approximate sizes.

There are also two things that you can do. First, on the machine where you are indexing the files, take a look at
$SPLUNK_HOME/var/log/splunk/splunkd.log
This is a lengthy log file, with a lot of different things in it. But at some point you will see where Splunk is setting up the tailing processor to follow the files that it is monitoring. If Splunk was unable to read one of the files, or if some other problem occurred, you should see it here. Look especially for anything labelled as an ERROR or WARNING.

Second, on the machine where you are collecting the data, you can run the Splunk command: splunk list monitor
This command will tell you which directories and files that Splunk is currently monitoring.

[instigardo]

Hi
Thanks for the reply, I checked the log file and there were few errors like this that I am getting.

10-11-2016 15:54:28.931 -0400 ERROR TailReader - File will not be read, seekptr checksum did not match (file=/data/logs/N00049260_20160927_093002-20160927_093208.log).  Last time we saw this initcrc, filename was different.  You may wish to use larger initCrcLen for this sourcetype, or a CRC salt on this source.  Consult the documentation or file a support case online at http://www.splunk.com/page/submit_issue for more info.

[ppablo]

HI @instigardo

The community can better help you troubleshoot your issue if you can share your monitor configuration from inputs.conf for this particular folder. Always try to include as much information about your environment and issue to save a back and forth comment thread asking for more details.