Hello Splunkers ,
I wrote a python script that explores the splunk-var indexes and calculates their total size, and then asks the user if they’d like to back it up.
After the user indicates which indexes they’d like to back up, it copies all buckets and other metadata in the db path (excluding the hot bucket) to a dir that is specified as a command line arg.
I want to know
Thanks in Advance
Hi @power12 ,
answering to your questions:
How to actually back up files (is it as simple as copying out the dir and then later copying it in and restarting splunk)
Best implement bucket policies (maxHotSpanSecs)
Understand bucket rollover when we have unexpected behavior
What indexes.conf should I use to have the bucket have one day worth of data
Ciao.
Giuseppe
Hi @power12,
as also @richgalloway said, it isn't a good idea having one day worth data because, unless you have very large data volumes (as not in your case), in this way you'll have a largen number of very small buckets.
leave the default values!
Ciao.
Giuseppe
Splunk has a document that explains how to backup and recover your indexes. It also explains rollover. See https://docs.splunk.com/Documentation/Splunk/9.0.3/Indexer/Backupindexeddata
We don't have enough information to answer the second question. What's best for you may not be best for others. Also, different indexes within a site may need different settings.
For a bucket to contain at most one day of data, set maxHotSpanSecs to 86400. Note that a bucket may contain less than a day of data if it filled up early.
@richgalloway Thank you for your reply . Yes I used the same setting but its chunking before 86400 ..so I checked with btool and saw that the maxDataSize is set to default which is 750 MB..changing that solved the issue
We have a single instance splunk and 100GB license and on an average we get 10GB of data per day..Even with this..is it not good practice to have 1 day worth data ?
I'm glad to hear you solved your problem.
I think it is not correct to say that 1 day of data is not good practice. It can be good practice, depending on your needs. Many sites use that practice to help ensure their data freezes in a timely fashion. If a bucket contains multiple days of data then old data in that bucket will remain searchable until the newest event in the bucket expires. That could violate the site's data retention policy.