Splunk Search

Is it possible to search cold bucket data only for a given index?

mendesjo
Path Finder

Hello, I modified my cold bucket location, and I want to perform some test queries for data residing in cold buckets storage only. Is there a way to do this?

Tags (1)
0 Karma

somesoni2
Revered Legend

One option could be to test with the time range of the cold buckets. Log on to the indexer and go to the $SPLUNK_DB/yourIndexName/colddb folder and pickup some buckets and get the timerange for the data they store. The bucket name would like db_epochLatestTime_epochEarliestTime_somecounter. Then you can run searches like this

index=yourindex earliest=epochEarliestTime latest=epochLatestTime
0 Karma

mendesjo
Path Finder

Thank you! Ok, that seemed to work but this is what I do not understand. I'm querying for data sitting in our old cold directory using the time stamp and i'm getting results, how? What I was expecting is that I would get no results, then manually copy the data to new cold directory as specified then re-try the query and I would get results, in order to test moving data from old cold directory to new.

0 Karma

somesoni2
Revered Legend

Did you make changes to your indexes.conf to update the colddb location and restarted the Splunk Indexer(s)??

0 Karma

mendesjo
Path Finder

Yes.. and after I restarted splunk I think newly created diretories and data being created in my new cold bucket location. Simply, as I stated what I would like to do is query for data that I did not copy over (with the expected result of no data retrieved for that time period), then manually copy cold bucket data from old location to new location, re-query and now see the data.

0 Karma

somesoni2
Revered Legend

Ok.. Gotcha.. The problem is that a bucket with name db_epochLatestTime_epochEarliestTime_somecounte doesn't contain all the events from period epochLatestTime and epochEarliestTime. There will be overlap between data for a day/period spread across multiple buckets. What you can do is pickup a bucket in old location, open the rawdata file (compressed file), zcat first few lines, find the keywords for that data and search in Splunk (you need to look for some kind of primary key. When bucket is in old location, you should see the data for that primary key and should see once they are copied over.

0 Karma
Get Updates on the Splunk Community!

New Dates, New City: Save the Date for .conf25!

Wake up, babe! New .conf25 dates AND location just dropped!! That's right, this year, .conf25 is taking place ...

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...