Splunk Enterprise

How to get archived data from frozen buckets in clustered indexers?

sanglap666
Loves-to-Learn

Hi All,

I want to get  Archived data from Frozen buckets for a certain time frame.
The index which i am trying to fetch is related to windows event logs.
Is their any script available to achive this in clustered environment.

Help in this is much appreciated!

Regards,

Sanglap

Labels (1)
0 Karma

woodcock
Esteemed Legend

If you didn't setup a frozen policy then your frozen buckets got deleted.
If you did, then it is pretty simple.
Just make sure that you have a thawed directory defined for your index.
Then pick an indexer and just drop the files there and that indexer should start searching them.

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

Splunk didn't offer any script for that, but you could do your own as many of us have done. Unfortunately I haven't any own as usually did those on customers environments. But with quickly searching I could found at least two which you can use at least as a starting point.

I haven't test those so check how those are working before using those on production.

r. Ismo.

0 Karma

yeahnah
Motivator

Hi @sanglap666 

You've not described what you frozen index policy does so you should start with sharing that as it could be a bespoke setup.  With saying that, I'm not aware of a thawing script as it really depends on what your frozen policy has done with the data/buckets anyway.  

Here's the the Splunk docs on thawing frozen/archived indexes.

https://docs.splunk.com/Documentation/Splunk/latest/Indexer/Restorearchiveddata

Basically, you can copy the archived data into the thaweddb directory of you indexer peers where Splunk can search it again. 

 $SPLUNK_HOME/var/lib/splunk/<your index>/thaweddb 

 Hope that helps get you started

0 Karma

sanglap666
Loves-to-Learn

HI @yeahnah ,

This is my policy

[wineventlog]
homePath.maxDataSizeMB = 50000
maxDataSize = auto
maxHotBuckets = 3
repFactor=auto
homePath = $SPLUNK_DB/wineventlog/db
coldPath = /xxxxxxx/splunk/var/lib/wineventlog/colddb
thawedPath = /xxxxx/splunk/var/lib/wineventlog/thaweddb
coldToFrozenDir = /yyyyyy/splunk/var/lib/wineventlog/frozendb
## 60 days in hot
maxHotSpanSecs = 5184000
## 4 months in cold
frozenTimePeriodInSecs = 10368000
tstatsHomePath = volume:_splunk_summaries/wineventlog/datamodel_summary

 

I see the data in frozen db also i can put it in thawed db as you recommended, but i only want the data in certain time frame say in between jan 2022 to july 2022.

Also i have replicated buckets(rb_...) as well does it needs to be put in the correct indexers based on guid to rebuild it again ?
How can i achive this?

0 Karma

yeahnah
Motivator

Hi @sanglap666 

The index bucket file names use this naming convention db_<newest_time>_<oldest_time>_<localid>_<guid> where the time is in epoch seconds.  Doc ref here...

https://docs.splunk.com/Documentation/Splunk/9.0.4/Indexer/HowSplunkstoresindexes#Bucket_names

Using a Splunk query like this may work, but I'm not 100% sure if state=frozen works or not.

| dbinspect index=wineventlog state=frozen
| eval startDate=strftime(startEpoch,"%A %d %B %Y %H:%M:%S")
| eval endDate=strftime(endEpoch,"%A %d %B %Y %H:%M:%S")
| fields index, path, startDate, endDate, state

 No, you should not need to copy the rb_* buckets and no you should be able to copy the buckets on any indexer and then be able to query the thawed data.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...