topic Re: Unable to archive frozen data to s3 in Getting Data In

Unable to archive frozen data to s3

basu42002 — Thu, 14 Dec 2017 23:39:46 GMT

Hello Everyone,

I am using the below in indexes.conf file, but the script never got executed instead the frozen files are deleted.
frozenTimePeriodInSecs = 1382400
coldToFrozenScript = "/opt/splunk/bin/python" "/opt/splunk/etc/apps/atl-cold-to-frozen-s3/bin/coldToFrozenS3.py"

Do I need to create sub folders on S3 bucket? because manually executing the script, without the sub folders works.

However if I manually execute the script something like "python coldtofrozens3.py arguments", its copying the data to s3.
Also I have tried coldToFrozenDir=, which is working.
But the coldtofrozen script never works. I am unable to test the script, as i am losing the frozen data.

Can some one please help/suggest what is going wrong here.

Re: Unable to archive frozen data to s3

micahkemp — Fri, 15 Dec 2017 00:04:00 GMT

Have you searched your internal splunk logs for any reference to the script? This may show you that the script is or is not running, and if it's producing any errors.

Did you restart Splunk after making the change to indexes.conf?

Re: Unable to archive frozen data to s3

jesse_corray — Fri, 15 Dec 2017 00:10:36 GMT

When you run it manually, are you using the same account that splunk runs as? If not, check the permissions.

Re: Unable to archive frozen data to s3

basu42002 — Fri, 15 Dec 2017 00:16:39 GMT

12-14-2017 23:44:43.621 +0000 INFO IndexWriter - idx=, Initializing, params='[300,period=60,frozenTimePeriodInSecs=1382400,coldToFrozenScript="/opt/splunk/bin/python" "/opt/splunk/etc/apps/atl-cold-to-frozen-s3/bin/coldToFrozenS3.py"

this is from the splunkd.log, I have restarted splunkindexer.

The changes are reflected, I see this script is being referenced in the log after restart.
However the script never executed.

Re: Unable to archive frozen data to s3

basu42002 — Fri, 15 Dec 2017 00:25:45 GMT

I don't see any issues with permissions, also I am using the same account that splunk is running as.

Re: Unable to archive frozen data to s3

basu42002 — Fri, 15 Dec 2017 00:27:19 GMT

Any suggestions please.
I have tried with and without double quotes:

"/opt/splunk/bin/python" "/opt/splunk/etc/apps/atl-cold-to-frozen-s3/bin/coldToFrozenS3.py"

But it did not work.

Re: Unable to archive frozen data to s3

basu42002 — Fri, 15 Dec 2017 05:07:40 GMT

From the logs, I see
ERROR BucketMover - coldToFrozenScript /usr/bin/python: /opt/splunk/lib/libssl.so.1.0.0: version `OPENSSL_1.0.0' not found (required by /usr/bin/python)

Is it possible to help in resolving this problem.

Re: Unable to archive frozen data to s3

micahkemp — Fri, 15 Dec 2017 12:58:50 GMT

When you test the app manually you should use Splunk's python interpreter:

./bin/splunk cmd python <script name>

Re: Unable to archive frozen data to s3

basu42002 — Fri, 15 Dec 2017 19:20:53 GMT

Thank you, if I run it manually it is working, but otherwise it doesn't work when splunk runs automatically:
./splunk cmd /opt/splunk/bin/python /opt/splunk/etc/apps/atl-cold-to-frozen-s3/bin/coldToFrozenS3.py /opt/splunkindexes/xxx/cold/testbk_bkt

Re: Unable to archive frozen data to s3

basu42002 — Fri, 15 Dec 2017 20:35:26 GMT

Looks like this is permission issue , as splunk user it is not able to execute was
can't open file '/usr/local/bin/aws': [Errno 13] Permission denied

Re: Unable to archive frozen data to s3

micahkemp — Thu, 28 Dec 2017 22:05:07 GMT

Were you able to solve the permission issue?

Re: Unable to archive frozen data to s3

basu42002 — Thu, 28 Dec 2017 23:01:10 GMT

We have decided to upgrade splunk to 7.0, as splunk is unable to execute aws. We couldn’t resolve permission issues.

Re: Unable to archive frozen data to s3

micahkemp — Thu, 28 Dec 2017 23:08:59 GMT

It doesn't seem likely that the Splunk version would cause (or remedy) the issue you were having with permissions.

Re: Unable to archive frozen data to s3

basu42002 — Thu, 28 Dec 2017 23:18:13 GMT

Splunk 7 has support for S3, so that’s the reason for upgrade. We are not going to use the coldtofrozen script anymore

Re: Unable to archive frozen data to s3

klaxdal — Mon, 08 Jan 2018 17:55:53 GMT

@ basu42002 ,

Can you explain how you set this up?

Thanks

Re: Unable to archive frozen data to s3

pccl — Thu, 22 Feb 2018 15:58:28 GMT

I'd also be interested to know how you got on with this. It's currently an unsupported feature and I'm unclear how to set it up.

Re: Unable to archive frozen data to s3

basu42002 — Tue, 29 Sep 2020 18:20:38 GMT

We have upgraded splunk indexer to 7.x and enabled boot start as Splunk user after changing the owner to splunk for $SPLUNKHOME
And finally able to automate cold to frozen s3.

The following worked as splunk user:
./splunk cmd /opt/splunk/bin/python /opt/splunk/etc/apps/atl-cold-to-frozen-s3/bin/coldToFrozenS3.py /opt/splunkindexes//cold/testbk_rb_6D93D52F011A

and we added the following under /opt/splunk/etc/slave-apps/_cluster/local
indexes.conf:
frozenTimePeriodInSecs = 1382400
coldToFrozenScript = "/opt/splunk/bin/python" "/opt/splunk/etc/apps/atl-cold-to-frozen-s3/bin/coldToFrozenS3.py"

following by splunk indexer restart

Re: Unable to archive frozen data to s3

markhill1 — Sun, 10 Jun 2018 22:46:25 GMT

Please let us know how you did this. Thanks!!

Re: Unable to archive frozen data to s3

basu42002 — Mon, 11 Jun 2018 02:16:16 GMT

I have used cold to frozen s3 script to achieve this. The steps are mentioned in the 'answer' section above. However we have upgraded to splunk 7.x as coldtofrozens3 script did not work in splunk 6.x