I am trying to forward a csv file to a new index. However, I found that it has already been forwarded to another index.
In inputs.conf, I removed the stanza pointing this file to the other index and added a new stanza pointing to the desired index and restarted the forwarder. The data is not present in the new index.
Is there a way to re-index this csv file into the new index?
The behavior your experiencing is due to the fishbucket on the forwarder. This keeps track of whats already been indexed to ensure there's no duplicate indexed data. You have a few options here, you can wipe the record from the fishbbucket, change the file name, or open the file and add/change a single character so it looks like a new file.
https://www.splunk.com/blog/2008/08/14/what-is-this-fishbucket-thing.html
https://www.splunk.com/blog/2008/08/27/more-fishbucket-fun.html
The behavior your experiencing is due to the fishbucket on the forwarder. This keeps track of whats already been indexed to ensure there's no duplicate indexed data. You have a few options here, you can wipe the record from the fishbbucket, change the file name, or open the file and add/change a single character so it looks like a new file.
https://www.splunk.com/blog/2008/08/14/what-is-this-fishbucket-thing.html
https://www.splunk.com/blog/2008/08/27/more-fishbucket-fun.html
How are you able to remove a single record from the fishbucket?
Thanks - I found that answer right after posting but couldn't find this as posted yet to be able to update- thanks again
Feel free to upvote if this answer helped you 🙂
Logon to the CLI of your UF and go to the splunk directory and issue this command. Make sure to replace $FILE with your file name
./splunk cmd btprobe -d $SPLUNK_HOME/var/lib/splunk/fishbucket/splunk_private_db --file $FILE --reset
Thanks! That did the trick