I'm trying to figure out how to analyze and manage specific records in the _fishbucket index.
I have big directories with many files splunk is monitoring, and our only method for reindexing right now is either cleaning the _fishbucket or adding crcSalts, both not very good for most use cases.
I want to be able to handle specific records in the _fishbucket - looking at this old blog post (what-is-this-fishbucket-thing) - it starts off by saying
To see what’s there, try searching for “index=_thefishbucket”
This simply doesn't work. And I want to also be able to delete specific records.
What's up with this? something changed since this blog post?
Yes, something changed since 2008! In fact, Andrea added a footnote to the end of the article in 2010: "Note that this old post only applies to 3.x versions"
Splunk no longer lets you look at the fishbucket index. You cannot manage the specific records. The format is not published and the files are kept in binary.
Thanks for the update. I guess I am a bit behind on the cool tools! Although it is still true that you can't just go editing or viewing the fishbucket!
Here is a link to the relevant section in the Troubleshooting Manual.
Are you wanting to delete certain records so that Splunk will (re-)index the files?
Note that you can issue
splunk add oneshot from the command line to prompt Splunk to index any file without regard for the fishbucket.
There is a tool to selectively forgot a single file from the fishbucket
./splunk cmd btprobe -d $SPLUNK_HOME/var/lib/splunk/fishbucket/splunk_private_db --file $FILE --reset
see options for btprobe
There are 2 possible ways to invoke this tool:
1: btprobe [-h or --help] -d <btree directory> [-k <hex key OR ALL> | --file <filename>] [--salt <salt>] [--validate] [--reset] [--bytes <bytes>] Queries the specified BTree for the given key or file. -d Directory that contains the btree index. (Required) -k Hex crc key or ALL to get all the keys. --file File to compute the crc from. (One of -k and --file must be specified. --validate Validate the btree to look for errors. --salt Salt the crc if --file param is specified. --reset Reset the fishbucket for the given key or file in the btree. --bytes Number of bytes to read when calculating CRC (default 256). 2: btprobe [-h or --help] --compute-crc <filename> [--salt <salt>] [--bytes <bytes>] Computes a crc from the specified file (using the given salt if any). Examples: btprobe -d /opt/splunk/var/lib/splunk/fishbucket/splunk_private_db -k 0xe8d117ddba85e714 --validate btprobe -d /opt/splunk/var/lib/splunk/fishbucket/splunk_private_db --file /var/log/inputfile --salt SOME_SALT btprobe --compute-crc /var/log/inputfile --salt SOME_SALT