Getting Data In

Unable to archive frozen data to s3

basu42002
Path Finder

Hello Everyone,

I am using the below in indexes.conf file, but the script never got executed instead the frozen files are deleted.
frozenTimePeriodInSecs = 1382400
coldToFrozenScript = "/opt/splunk/bin/python" "/opt/splunk/etc/apps/atl-cold-to-frozen-s3/bin/coldToFrozenS3.py"

Do I need to create sub folders on S3 bucket? because manually executing the script, without the sub folders works.

However if I manually execute the script something like "python coldtofrozens3.py arguments", its copying the data to s3.
Also I have tried coldToFrozenDir=, which is working.
But the coldtofrozen script never works. I am unable to test the script, as i am losing the frozen data.

Can some one please help/suggest what is going wrong here.

0 Karma
1 Solution

basu42002
Path Finder

We have upgraded splunk indexer to 7.x and enabled boot start as Splunk user after changing the owner to splunk for $SPLUNKHOME
And finally able to automate cold to frozen s3.

The following worked as splunk user:
./splunk cmd /opt/splunk/bin/python /opt/splunk/etc/apps/atl-cold-to-frozen-s3/bin/coldToFrozenS3.py /opt/splunkindexes//cold/testbk_rb_6D93D52F011A

and we added the following under /opt/splunk/etc/slave-apps/_cluster/local
indexes.conf:
frozenTimePeriodInSecs = 1382400
coldToFrozenScript = "/opt/splunk/bin/python" "/opt/splunk/etc/apps/atl-cold-to-frozen-s3/bin/coldToFrozenS3.py"

following by splunk indexer restart

View solution in original post

0 Karma

basu42002
Path Finder

We have upgraded splunk indexer to 7.x and enabled boot start as Splunk user after changing the owner to splunk for $SPLUNKHOME
And finally able to automate cold to frozen s3.

The following worked as splunk user:
./splunk cmd /opt/splunk/bin/python /opt/splunk/etc/apps/atl-cold-to-frozen-s3/bin/coldToFrozenS3.py /opt/splunkindexes//cold/testbk_rb_6D93D52F011A

and we added the following under /opt/splunk/etc/slave-apps/_cluster/local
indexes.conf:
frozenTimePeriodInSecs = 1382400
coldToFrozenScript = "/opt/splunk/bin/python" "/opt/splunk/etc/apps/atl-cold-to-frozen-s3/bin/coldToFrozenS3.py"

following by splunk indexer restart

0 Karma

basu42002
Path Finder

We have decided to upgrade splunk to 7.0, as splunk is unable to execute aws. We couldn’t resolve permission issues.

0 Karma

micahkemp
Champion

It doesn't seem likely that the Splunk version would cause (or remedy) the issue you were having with permissions.

0 Karma

basu42002
Path Finder

Splunk 7 has support for S3, so that’s the reason for upgrade. We are not going to use the coldtofrozen script anymore

0 Karma

markhill1
Path Finder

Please let us know how you did this. Thanks!!

basu42002
Path Finder

I have used cold to frozen s3 script to achieve this. The steps are mentioned in the 'answer' section above. However we have upgraded to splunk 7.x as coldtofrozens3 script did not work in splunk 6.x

0 Karma

pccl
Engager

I'd also be interested to know how you got on with this. It's currently an unsupported feature and I'm unclear how to set it up.

klaxdal
Contributor

@ basu42002 ,

Can you explain how you set this up?

Thanks

micahkemp
Champion

Have you searched your internal splunk logs for any reference to the script? This may show you that the script is or is not running, and if it's producing any errors.

Did you restart Splunk after making the change to indexes.conf?

0 Karma

basu42002
Path Finder

12-14-2017 23:44:43.621 +0000 INFO IndexWriter - idx=, Initializing, params='[300,period=60,frozenTimePeriodInSecs=1382400,coldToFrozenScript="/opt/splunk/bin/python" "/opt/splunk/etc/apps/atl-cold-to-frozen-s3/bin/coldToFrozenS3.py"

this is from the splunkd.log, I have restarted splunkindexer.

The changes are reflected, I see this script is being referenced in the log after restart.
However the script never executed.

0 Karma

basu42002
Path Finder

From the logs, I see
ERROR BucketMover - coldToFrozenScript /usr/bin/python: /opt/splunk/lib/libssl.so.1.0.0: version `OPENSSL_1.0.0' not found (required by /usr/bin/python)

Is it possible to help in resolving this problem.

0 Karma

micahkemp
Champion

When you test the app manually you should use Splunk's python interpreter:

./bin/splunk cmd python <script name>
0 Karma

basu42002
Path Finder

Thank you, if I run it manually it is working, but otherwise it doesn't work when splunk runs automatically:
./splunk cmd /opt/splunk/bin/python /opt/splunk/etc/apps/atl-cold-to-frozen-s3/bin/coldToFrozenS3.py /opt/splunkindexes/xxx/cold/testbk_bkt

0 Karma

basu42002
Path Finder

Looks like this is permission issue , as splunk user it is not able to execute was
can't open file '/usr/local/bin/aws': [Errno 13] Permission denied

0 Karma

micahkemp
Champion

Were you able to solve the permission issue?

0 Karma

jesse_corray
Explorer

When you run it manually, are you using the same account that splunk runs as? If not, check the permissions.

0 Karma

basu42002
Path Finder

I don't see any issues with permissions, also I am using the same account that splunk is running as.

0 Karma

basu42002
Path Finder

Any suggestions please.
I have tried with and without double quotes:

"/opt/splunk/bin/python" "/opt/splunk/etc/apps/atl-cold-to-frozen-s3/bin/coldToFrozenS3.py"

But it did not work.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...