I have a folder that consists of logs added every day i have given this folder as input to splunk to continuously monitor the folder and do few actions as per the search query but splunk is not picking up the newly indexed data but when i append the new data to existing files splunk starts picking up can anyone help me with this
Are you sure that your monitor stanza is picking up the new files?
Is it only monitoring a folder? Check the splunkd.log for details on what tailing processor is doing.
For more details
1) Go to the splunk forwarder installation and edit the $SPLUNK_HOME/etc/log.cfg
2) Find these components and change them to DEBUG
TailingProcessor. BatchReader, WatchedFile
3) Restart the forwarder
4) Look for which files are being read
You can also hit the REST endpoint at port 8089 and look at file input status