Getting Data In

Intermediate throwaway index

ddrillic
Ultra Champion

In order to validate all the configurations prior to using the real index for a certain customer, we decided to use a temporary index called throwaway. Upon validation of the data, we switch the configurations to point to the real index. However, we reach situations where there is no new data for this index and it's tough then to present to the customer the finished product. In addition, using ignoreOlderThan = 7 for the throwaway index and when switching, we pick up only the new data. We apply this method for hundreds of internal customers and I wonder if and how the method can be improved...

Tags (1)
0 Karma
1 Solution

pradeepkumarg
Influencer

two suggestions

  1. If you have a non prod splunk instance, you should try testing in that rather than your production instance. Non prod servers sending data to non prod splunk instance to test your configs.
  2. If you absolutely have to test in prod in a throwaway index, you should probably clear the fish bucket on the forwarders every time you change the index to real index. https://www.splunk.com/blog/2008/08/14/what-is-this-fishbucket-thing.html

View solution in original post

sudosplunk
Motivator

Hi,

Did you try using crcSalt or initCrcLength in your inputs.conf? I mean, if your goal is to re-index data each time you want to test your configurations, then use these settings with appropriate values and change your sourcetype to find difference.

This will save you a restart of the indexer.

0 Karma

darrenfuller
Contributor

Rather than creating inputs.conf definitions to push your files to the throwaway index, which as others have mentioned will add the filename to the fishbucket and then require you to take action to get those files re-indexed again. Add the files into splunk using oneshot pointing to the throwaway index and then review the data as it has been parsed. lather rinse repeat until your sourcetypes are working correctly.

$SPLUNK_HOME/bin/splunk add oneshot /path/to/file.txt -index throwaway -sourcetype mynewtestingsourcetype -source testrun-24

If you customize the source each time you run a test, it makes it easier to separate the previously oneshotted data from the new

0 Karma

pradeepkumarg
Influencer

two suggestions

  1. If you have a non prod splunk instance, you should try testing in that rather than your production instance. Non prod servers sending data to non prod splunk instance to test your configs.
  2. If you absolutely have to test in prod in a throwaway index, you should probably clear the fish bucket on the forwarders every time you change the index to real index. https://www.splunk.com/blog/2008/08/14/what-is-this-fishbucket-thing.html

ddrillic
Ultra Champion

I agree with your points @gpradeepkumarreddy.

0 Karma

ddrillic
Ultra Champion

I wonder if there is a REST call to clear the fishbucket. Then we can invoke it from a script that iterates through all the servers which are involved.

0 Karma

ddrillic
Ultra Champion

@gpradeepkumarreddy - great suggestions !!!

0 Karma
Get Updates on the Splunk Community!

How to Monitor Google Kubernetes Engine (GKE)

We’ve looked at how to integrate Kubernetes environments with Splunk Observability Cloud, but what about ...

Index This | How can you make 45 using only 4?

October 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Splunk Education Goes to Washington | Splunk GovSummit 2024

If you’re in the Washington, D.C. area, this is your opportunity to take your career and Splunk skills to the ...