Getting Data In

coldToFrozenScript not working

sgramenopoulos
Explorer

Below is my indexes.conf file:

defaultDatabase = main

[main]

homePath= $SPLUNK_DB\defaultdb\db
coldPath = $SPLUNK_DB\defaultdb\colddb
thawedPath = $SPLUNK_DB\defaultdb\thawedb
maxDataSize = 5     
maxHotBuckets = 1                        
maxWarmDBCount = 1
frozenTimePeriodInSecs = 60
rotatePeriodInSecs = 60
coldToFrozenScript = WindowsCompressedExport.bat "$DIR"

I have the fields set to a minimum so I can see if this works.

Also here is my WindowsCompressedExport.bat file:

set dest_base=C:\Security\splunk\

set source_path=%1
set source_base=%~dp1
set source_leaf=%~nx1
set dest_final=%dest_base%\%source_leaf%

#echo commands....

for %%i iin (%1\*.tsidx) do splunk-compresstool.exe -M "%%i"

mkdir %dest_final%

xcopy %1 %dest_final% /E /I /C /Y

Any clue as to what may be the issue??

Tags (2)

sgramenopoulos
Explorer

Turns out it was the trailing "\" for the dest_base field in the WindowsCompressedExport.bat file.

0 Karma

dwaddle
SplunkTrust
SplunkTrust

(not a windows expert, so YMMV)

I would start by making sure Splunk "sees" your coldToFrozenScript argument. The btool diagnostic command can help, something like:

bin/splunk cmd btool --debug indexes list main

This will dump the "merged" (as Splunk would see/use it) configuration stanzas so you can be sure your stuff is being applied. This is a good to check to make sure you got your CamelCase right. (In your example, it looks like you have it correct, but I've seen folks get it wrong before.)

I'm also not sure about the "$DIR" part - it seems like it should work, but the quoting makes me a little nervous. If the path does not contain spaces, perhaps take the quotes off entirely - or as a test, hard-code the export path.

Also, as a troubleshooting step, you could have your script create a dummy file in a well known place (like, say, C:\temp) -- then you have some proof as to whether or not your script got called at all.

Finally, in your splunkd.log you should see messages similar to these - which fire on the beginning and successful end of a freeze operation.

01-10-2011 08:46:16.003 INFO  BucketMover - will attempt to freeze: /opt/splunk/var/lib/splunk/firewalls/db/db_1286891137_1286854127_366 because frozenTimePeriodInSecs=7776000 exceeds difference between now=1294667176 and latest=1286891137
01-10-2011 08:46:39.234 INFO  BucketMover - AsyncFreezer freeze succeeded for /opt/splunk/var/lib/splunk/firewalls/db/db_1286891137_1286854127_366

sgramenopoulos
Explorer

btool returned what I had set in the .conf file.

I referenced the Admin guide on usage of the ColdToFrozenScript as noted here:

coldToFrozenScript =

0 Karma

ftk
Motivator

Your script has the following line:

for %%i iin (%1\*.tsidx) do splunk-compresstool.exe -M "%%i"

Is the "iin" actually in the script? If so, that's a typo, it should read "in". This could cause your script to fail.

0 Karma

ftk
Motivator

Just a typo on answers or in your script? If in your script, you should fix it...

0 Karma

sgramenopoulos
Explorer

Yes this is typo.

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...