Splunk Newbie: Configuring hot and cold storage on 2 separate volumes (AWS)

[Chrisdarn]

Using Splunk Enterprise 6.5.3

Hello,
I have recently downloaded Splunk Enterprise on an AWS linux instance and have mounted a fast volume and and a large storage volume. These are the following folders, with test1 as the index name:
/data/hot/test1
/data/cold/test1
The fast volume has a mountpoint of /data/hot and large storage as /data/cold.
I've added "SPLUNK_DB=/data" in the /etc/environment file and the /opt/splunk/etc/splunk-launch.conf file

I copied over the buckets from /opt/splunk/var/lib/splunk/defaultdb/ to these new folders:
sudo cp /opt/splunk/var/lib/splunk/defaultdb/db /data/hot/test1/
sudo cp /opt/splunk/var/lib/splunk/defaultdb/colddb /data/cold/test1/
sudo cp /opt/splunk/var/lib/splunk/defaultdb/thaweddb /data/cold/test1/
So now the following folders exist:
/data/hot/test1/db
/data/cold/test1/colddb
/data/cold/test1/thaweddb

The problem is I've created a custom indexes.conf file in /opt/splunk/etc/local/indexes.conf and what I want it to do is at a certain size move the data from hot to warm and when it reaches a certain time, move to cold storage. I've gone through the documentation, edited this file a whole bunch of times (and restarted it after edits) and I still cannot figure out how I can move this data between buckets. I've also read in some places that you cannot move from warm to cold by time, which in that case, how would you do it by size?

I've tested by uploading a zip file to the Splunk service on :8000. This then creates a hot bucket on /data/hot/test1/db/hot_v1_0 After I restart the service, this then converts this hot bucket to a warm bucket (I'm assuming) on /data/hot/test1/db/db_mostrecenteventtime_latesteventtime). A prerequisite was also to create a new index on the web interface called test1 for Splunk to be able to store it there, where I've added the /data/hot etc and /data/cold etc directories.

As I'm new to Splunk I'm not 100% sure if that's how it can work, so if anyone can help on this issue/ explain Splunk to me, that would be greatly, greatly appreciated!

[adonio]

Hello Chrisdarn,
it is not exactly clear to me if the folders exists are created by the index or by you.
in any case, lets assume i created 2 directories:
/hotWarm/buckets/directory
/cold/buckets/directory.
I will use indexes.conf to create the index, lets call it bob with its directories and rules.
an explanation for each config is under each line:

[bob]
homePath   = /hotWarm/buckets/directory/bob/db
#where hot buckets will be written to.
coldPath   = /cold/buckets/directory/bob/colddb
#where cold buckets will be written to.
thawedPath = /cold/buckets/directory/bob/thaweddb
#An absolute path that contains the thawed (resurrected) databases for the  index 
homePath.maxDataSizeMB = 4000
# sets the size limit for the hotWarm folder
coldPath.maxDataSizeMB = 6000
# sets the size limit for the cold folder
maxTotalDataSizeMB = 10000
# total size of the index (hotWarm + cold)
maxDataSize = auto
# The maximum size in MB for a hot DB to reach before a roll to warm is  triggered
frozenTimePeriodInSecs = 103680000
# Number of seconds after which indexed data rolls to frozen 

full details on indexes.conf and more examples, here: https://docs.splunk.com/Documentation/Splunk/6.5.3/Admin/Indexesconf

hope it helps

[Chrisdarn]

Thanks for the help so far. All the folders in /opt/splunk and its subdirectories were there when I first installed Splunk Enterprise. I've just copied these buckets and put them in my own volume.

[adonio]

hmm, it means that splunk will still write to old location.
you have to tell splunk where to write the data too
i will highly suggest not to just copy buckets from place to place regardless.

[Chrisdarn]

Okay I will note that for future reference. Where would I configure where to write the data to? I created an index on the Web UI to point to these new locations if that's what you mean.

[brreeves_splunk]

Here's a fun, but unsupported tool!

You can slide the sliders around to lay out what you'd like Splunk to do, and it'll give you configuration stanzas to apply to indexes.conf

http://splunk-sizing.appspot.com/

It's unsupported because it's not written/controlled by us. But it's usually pretty accurate.

[Chrisdarn]

Thanks for the link. I will try and see if this helps.