https://community.splunk.com/t5/Getting-Data-In/HadoopConnect-How-do-I-reset-the-HDFS-input/m-p/89680#M18615 <P>I have a folder in HDFS that has log files continuously being put into it. I decided to test the HadoopConnect app's import feature and created a test index to store data. Then, I added the folder to the input via the web interface on HadoopConnect. The data imports successfully, and new files are being indexed correctly too. </P> <P>I decide that I would like to use the app, and delete the test index and input. Then, I add the same input again. However, it seems that there is some persistency in the HDFS file monitoring because <STRONG>only new files are getting indexed</STRONG>. The old ones aren't anymore. </P> <P>I'd like to know if there is a way to reset this persistent state? I tried deleting <CODE>$SPLUNK_HOME/var/lib/splunk/persistentstorage/fschangemanager_state</CODE> because it seemed like a good candidate but to no avail. Please advise, thanks.</P> Tue, 09 Jul 2013 18:55:19 GMT jonahtang 2013-07-09T18:55:19Z https://community.splunk.com/t5/Getting-Data-In/HadoopConnect-How-do-I-reset-the-HDFS-input/m-p/89680#M18615 <P>I have a folder in HDFS that has log files continuously being put into it. I decided to test the HadoopConnect app's import feature and created a test index to store data. Then, I added the folder to the input via the web interface on HadoopConnect. The data imports successfully, and new files are being indexed correctly too. </P> <P>I decide that I would like to use the app, and delete the test index and input. Then, I add the same input again. However, it seems that there is some persistency in the HDFS file monitoring because <STRONG>only new files are getting indexed</STRONG>. The old ones aren't anymore. </P> <P>I'd like to know if there is a way to reset this persistent state? I tried deleting <CODE>$SPLUNK_HOME/var/lib/splunk/persistentstorage/fschangemanager_state</CODE> because it seemed like a good candidate but to no avail. Please advise, thanks.</P> Tue, 09 Jul 2013 18:55:19 GMT https://community.splunk.com/t5/Getting-Data-In/HadoopConnect-How-do-I-reset-the-HDFS-input/m-p/89680#M18615 jonahtang 2013-07-09T18:55:19Z https://community.splunk.com/t5/Getting-Data-In/HadoopConnect-How-do-I-reset-the-HDFS-input/m-p/89681#M18616 <P>Found it: The state data is in <CODE>$SPLUNK_HOME/var/lib/splunk/modinputs/hdfs</CODE>. Deleting the file(s) in this seems to make splunk index everything again.</P> Tue, 09 Jul 2013 21:22:43 GMT https://community.splunk.com/t5/Getting-Data-In/HadoopConnect-How-do-I-reset-the-HDFS-input/m-p/89681#M18616 jonahtang 2013-07-09T21:22:43Z