topic Re: Intermediate throwaway index in Getting Data In

Intermediate throwaway index

ddrillic — Thu, 16 Aug 2018 16:43:18 GMT

In order to validate all the configurations prior to using the real index for a certain customer, we decided to use a temporary index called throwaway. Upon validation of the data, we switch the configurations to point to the real index. However, we reach situations where there is no new data for this index and it's tough then to present to the customer the finished product. In addition, using ignoreOlderThan = 7 for the throwaway index and when switching, we pick up only the new data. We apply this method for hundreds of internal customers and I wonder if and how the method can be improved...

Re: Intermediate throwaway index

pradeepkumarg — Thu, 16 Aug 2018 17:04:09 GMT

two suggestions

If you have a non prod splunk instance, you should try testing in that rather than your production instance. Non prod servers sending data to non prod splunk instance to test your configs.
If you absolutely have to test in prod in a throwaway index, you should probably clear the fish bucket on the forwarders every time you change the index to real index. https://www.splunk.com/blog/2008/08/14/what-is-this-fishbucket-thing.html

Re: Intermediate throwaway index

ddrillic — Thu, 16 Aug 2018 18:24:11 GMT

I agree with your points @gpradeepkumarreddy.

Re: Intermediate throwaway index

darrenfuller — Thu, 16 Aug 2018 19:13:46 GMT

Rather than creating inputs.conf definitions to push your files to the throwaway index, which as others have mentioned will add the filename to the fishbucket and then require you to take action to get those files re-indexed again. Add the files into splunk using oneshot pointing to the throwaway index and then review the data as it has been parsed. lather rinse repeat until your sourcetypes are working correctly.

$SPLUNK_HOME/bin/splunk add oneshot /path/to/file.txt -index throwaway -sourcetype mynewtestingsourcetype -source testrun-24

If you customize the source each time you run a test, it makes it easier to separate the previously oneshotted data from the new

Re: Intermediate throwaway index

ddrillic — Fri, 17 Aug 2018 00:38:09 GMT

I wonder if there is a REST call to clear the fishbucket. Then we can invoke it from a script that iterates through all the servers which are involved.

Re: Intermediate throwaway index

sudosplunk — Fri, 17 Aug 2018 13:24:59 GMT

Hi,

Did you try using crcSalt or initCrcLength in your inputs.conf? I mean, if your goal is to re-index data each time you want to test your configurations, then use these settings with appropriate values and change your sourcetype to find difference.

This will save you a restart of the indexer.

Re: Intermediate throwaway index

ddrillic — Mon, 18 Feb 2019 00:20:12 GMT

@gpradeepkumarreddy - great suggestions !!!