Intermediate throwaway index

[ddrillic]

In order to validate all the configurations prior to using the real index for a certain customer, we decided to use a temporary index called throwaway. Upon validation of the data, we switch the configurations to point to the real index. However, we reach situations where there is no new data for this index and it's tough then to present to the customer the finished product. In addition, using ignoreOlderThan = 7 for the throwaway index and when switching, we pick up only the new data. We apply this method for hundreds of internal customers and I wonder if and how the method can be improved...

[sudosplunk]

Hi,

Did you try using crcSalt or initCrcLength in your inputs.conf? I mean, if your goal is to re-index data each time you want to test your configurations, then use these settings with appropriate values and change your sourcetype to find difference.

This will save you a restart of the indexer.

[darrenfuller]

Rather than creating inputs.conf definitions to push your files to the throwaway index, which as others have mentioned will add the filename to the fishbucket and then require you to take action to get those files re-indexed again. Add the files into splunk using oneshot pointing to the throwaway index and then review the data as it has been parsed. lather rinse repeat until your sourcetypes are working correctly.

$SPLUNK_HOME/bin/splunk add oneshot /path/to/file.txt -index throwaway -sourcetype mynewtestingsourcetype -source testrun-24

If you customize the source each time you run a test, it makes it easier to separate the previously oneshotted data from the new

[pradeepkumarg · Accepted solution]

two suggestions

1. If you have a non prod splunk instance, you should try testing in that rather than your production instance. Non prod servers sending data to non prod splunk instance to test your configs.
2. If you absolutely have to test in prod in a throwaway index, you should probably clear the fish bucket on the forwarders every time you change the index to real index. https://www.splunk.com/blog/2008/08/14/what-is-this-fishbucket-thing.html

[ddrillic]

I agree with your points @gpradeepkumarreddy.

[ddrillic]

I wonder if there is a REST call to clear the fishbucket. Then we can invoke it from a script that iterates through all the servers which are involved.

[ddrillic]

@gpradeepkumarreddy - great suggestions !!!