topic coldToFrozenScript not working in Getting Data In

coldToFrozenScript not working

sgramenopoulos — Tue, 11 Jan 2011 01:44:37 GMT

Below is my indexes.conf file:

defaultDatabase = main

[main]

homePath= $SPLUNK_DB\defaultdb\db
coldPath = $SPLUNK_DB\defaultdb\colddb
thawedPath = $SPLUNK_DB\defaultdb\thawedb
maxDataSize = 5     
maxHotBuckets = 1                        
maxWarmDBCount = 1
frozenTimePeriodInSecs = 60
rotatePeriodInSecs = 60
coldToFrozenScript = WindowsCompressedExport.bat "$DIR"

I have the fields set to a minimum so I can see if this works.

Also here is my WindowsCompressedExport.bat file:

set dest_base=C:\Security\splunk\

set source_path=%1
set source_base=%~dp1
set source_leaf=%~nx1
set dest_final=%dest_base%\%source_leaf%

#echo commands....

for %%i iin (%1\*.tsidx) do splunk-compresstool.exe -M "%%i"

mkdir %dest_final%

xcopy %1 %dest_final% /E /I /C /Y

Any clue as to what may be the issue??

Re: coldToFrozenScript not working

ftk — Tue, 11 Jan 2011 02:22:28 GMT

Your script has the following line:

for %%i iin (%1\*.tsidx) do splunk-compresstool.exe -M "%%i"

Is the "iin" actually in the script? If so, that's a typo, it should read "in". This could cause your script to fail.

Re: coldToFrozenScript not working

dwaddle — Tue, 11 Jan 2011 02:23:13 GMT

(not a windows expert, so YMMV)

I would start by making sure Splunk "sees" your coldToFrozenScript argument. The btool diagnostic command can help, something like:

bin/splunk cmd btool --debug indexes list main

This will dump the "merged" (as Splunk would see/use it) configuration stanzas so you can be sure your stuff is being applied. This is a good to check to make sure you got your CamelCase right. (In your example, it looks like you have it correct, but I've seen folks get it wrong before.)

I'm also not sure about the "$DIR" part - it seems like it should work, but the quoting makes me a little nervous. If the path does not contain spaces, perhaps take the quotes off entirely - or as a test, hard-code the export path.

Also, as a troubleshooting step, you could have your script create a dummy file in a well known place (like, say, C:\temp) -- then you have some proof as to whether or not your script got called at all.

Finally, in your splunkd.log you should see messages similar to these - which fire on the beginning and successful end of a freeze operation.

01-10-2011 08:46:16.003 INFO  BucketMover - will attempt to freeze: /opt/splunk/var/lib/splunk/firewalls/db/db_1286891137_1286854127_366 because frozenTimePeriodInSecs=7776000 exceeds difference between now=1294667176 and latest=1286891137
01-10-2011 08:46:39.234 INFO  BucketMover - AsyncFreezer freeze succeeded for /opt/splunk/var/lib/splunk/firewalls/db/db_1286891137_1286854127_366

Re: coldToFrozenScript not working

sgramenopoulos — Tue, 11 Jan 2011 02:31:21 GMT

Yes this is typo.

Re: coldToFrozenScript not working

ftk — Tue, 11 Jan 2011 02:35:16 GMT

Just a typo on answers or in your script? If in your script, you should fix it...

Re: coldToFrozenScript not working

sgramenopoulos — Tue, 11 Jan 2011 02:39:52 GMT

btool returned what I had set in the .conf file.

I referenced the Admin guide on usage of the ColdToFrozenScript as noted here:

coldToFrozenScript =

Re: coldToFrozenScript not working

sgramenopoulos — Tue, 11 Jan 2011 03:09:19 GMT

Turns out it was the trailing "\" for the dest_base field in the WindowsCompressedExport.bat file.