topic Re: Searching _fishbucket in Splunk Search

Searching _fishbucket

pembleton — Tue, 06 Aug 2013 14:50:09 GMT

I'm trying to figure out how to analyze and manage specific records in the _fishbucket index.

I have big directories with many files splunk is monitoring, and our only method for reindexing right now is either cleaning the _fishbucket or adding crcSalts, both not very good for most use cases.

I want to be able to handle specific records in the _fishbucket - looking at this old blog post (what-is-this-fishbucket-thing) - it starts off by saying

 To see what’s there, try searching for “index=_thefishbucket”

This simply doesn't work. And I want to also be able to delete specific records.

What's up with this? something changed since this blog post?

Re: Searching _fishbucket

lguinn2 — Tue, 06 Aug 2013 19:26:03 GMT

Yes, something changed since 2008! In fact, Andrea added a footnote to the end of the article in 2010: "Note that this old post only applies to 3.x versions"

Splunk no longer lets you look at the fishbucket index. You cannot manage the specific records. The format is not published and the files are kept in binary.

Thanks for the update. I guess I am a bit behind on the cool tools! Although it is still true that you can't just go editing or viewing the fishbucket!

Here is a link to the relevant section in the Troubleshooting Manual.

Re: Searching _fishbucket

sowings — Tue, 06 Aug 2013 19:42:10 GMT

Are you wanting to delete certain records so that Splunk will (re-)index the files?

Note that you can issue splunk add oneshot from the command line to prompt Splunk to index any file without regard for the fishbucket.

http://docs.splunk.com/Documentation/Splunk/5.0.1/Data/MonitorfilesanddirectoriesusingtheCLI

Re: Searching _fishbucket

yannK — Wed, 07 Aug 2013 00:47:33 GMT

There is a tool to selectively forgot a single file from the fishbucket

./splunk cmd btprobe -d $SPLUNK_HOME/var/lib/splunk/fishbucket/splunk_private_db --file $FILE --reset

see options for btprobe

There are 2 possible ways to invoke this tool:

 1: btprobe [-h or --help] -d <btree directory> [-k <hex key OR ALL> | --file <filename>] [--salt <salt>] [--validate] [--reset] [--bytes <bytes>]
 Queries the specified BTree for the given key or file.

    -d           Directory that contains the btree index. (Required)

    -k           Hex crc key or ALL to get all the keys.
    --file       File to compute the crc from.
      (One of -k and --file must be specified.

    --validate   Validate the btree to look for errors.
    --salt       Salt the crc if --file param is specified.
    --reset      Reset the fishbucket for the given key or file in the btree.
    --bytes      Number of bytes to read when calculating CRC (default 256).

 2: btprobe [-h or --help] --compute-crc <filename> [--salt <salt>] [--bytes <bytes>]
 Computes a crc from the specified file (using the given salt if any).

 Examples:
 btprobe -d /opt/splunk/var/lib/splunk/fishbucket/splunk_private_db  -k 0xe8d117ddba85e714 --validate
 btprobe -d /opt/splunk/var/lib/splunk/fishbucket/splunk_private_db --file /var/log/inputfile --salt SOME_SALT
 btprobe --compute-crc /var/log/inputfile --salt SOME_SALT