https://community.splunk.com/t5/Getting-Data-In/Monitor-vs-Batch-Job-for-inputting-data/m-p/185495#M37141 <P>Usually Batch input is used for large historical data. It is not a good practice for small and continuously updating files.</P> <P>Consider a case where i faced using batch input i fed splunk a large files and folders and in between the license limit was reached and stopped splunk to wait 24 hours and later i couldn't start indexing from where it stopped because the file was deleted. Due to <CODE>move_policy = sinkhole</CODE> policy that is mandatory for batch inputs. And I had no clue till what point the data was indexed all i did was i flushed the index and reindex whole data using monitor.</P> <P>It depends on the size of the data and its availability you should be able to choose the best one.</P> Mon, 11 May 2015 14:24:27 GMT krish3 2015-05-11T14:24:27Z https://community.splunk.com/t5/Getting-Data-In/Monitor-vs-Batch-Job-for-inputting-data/m-p/185494#M37140 <P>Currently, my preProd environment is set up to monitor logs from 100-150 servers with the monitor stanza in inputs.conf. I have been asked to research changing all these stanza to batch jobs because the pre-Prod forwarders apparently stop running from an overload of the saved files on them. I was wondering if anyone has had to tackle this before or if there is a better way of doing this. I figure the forwarder admins don't want to set up jobs on their own servers so they want Splunk to delete the files once indexed. </P> <P>Assistance would be appreciated. Thank you </P> Mon, 11 May 2015 14:06:30 GMT https://community.splunk.com/t5/Getting-Data-In/Monitor-vs-Batch-Job-for-inputting-data/m-p/185494#M37140 JoeSco27 2015-05-11T14:06:30Z https://community.splunk.com/t5/Getting-Data-In/Monitor-vs-Batch-Job-for-inputting-data/m-p/185495#M37141 <P>Usually Batch input is used for large historical data. It is not a good practice for small and continuously updating files.</P> <P>Consider a case where i faced using batch input i fed splunk a large files and folders and in between the license limit was reached and stopped splunk to wait 24 hours and later i couldn't start indexing from where it stopped because the file was deleted. Due to <CODE>move_policy = sinkhole</CODE> policy that is mandatory for batch inputs. And I had no clue till what point the data was indexed all i did was i flushed the index and reindex whole data using monitor.</P> <P>It depends on the size of the data and its availability you should be able to choose the best one.</P> Mon, 11 May 2015 14:24:27 GMT https://community.splunk.com/t5/Getting-Data-In/Monitor-vs-Batch-Job-for-inputting-data/m-p/185495#M37141 krish3 2015-05-11T14:24:27Z