Getting Data In

Splunk: Archive to S3 or S3 compatible Object Store using Hadoop ?

sambhram
New Member

I am seeing the following error message while trying to archive to S3. The logs are from "splunk_archiver.log". Any pointer as how to fix this ?

2019-09-09 06:09:11.127 -0700 ERROR Roller - Exception when deciding if bucket should be copied. bucket_name=db_1566816265_1543370757_53_D55DFE1B-5E59-4A40-A430-7A8334A5FE47, splunk_index=main, dir=/opt/splunk/var/lib/splunk/defaultdb/db/db_1566816265_1543370757_53_D55DFE1B-5E59-4A40-A430-7A8334A5FE47
2019-09-09 06:20:54.669 -0700 ERROR Roller - Exception when deciding if bucket should be copied. bucket_name=db_1553167044_1545418447_34_D55DFE1B-5E59-4A40-A430-7A8334A5FE47, splunk_index=main, dir=/opt/splunk/var/lib/splunk/defaultdb/db/db_1553167044_1545418447_34_D55DFE1B-5E59-4A40-A430-7A8334A5FE47
2019-09-09 06:33:11.420 -0700 ERROR Roller - Exception when deciding if bucket should be copied. bucket_name=db_1554020317_1546292628_98_D55DFE1B-5E59-4A40-A430-7A8334A5FE47, splunk_index=main, dir=/opt/splunk/var/lib/splunk/defaultdb/db/db_1554020317_1546292628_98_D55DFE1B-5E59-4A40-A430-7A8334A5FE47
2019-09-09 06:44:50.075 -0700 ERROR Roller - Exception when deciding if bucket should be copied. bucket_name=db_1549687163_1546343729_93_D55DFE1B-5E59-4A40-A430-7A8334A5FE47, splunk_index=main, dir=/opt/splunk/var/lib/splunk/defaultdb/db/db_1549687163_1546343729_93_D55DFE1B-5E59-4A40-A430-7A8334A5FE47
2019-09-09 06:57:14.182 -0700 ERROR Roller - Exception when deciding if bucket should be copied. bucket_name=db_1548954761_1546346643_1_D55DFE1B-5E59-4A40-A430-7A8334A5FE47, splunk_index=main, dir=/opt/splunk/var/lib/splunk/defaultdb/db/db_1548954761_1546346643_1_D55DFE1B-5E59-4A40-A430-7A8334A5FE47

0 Karma

rdagan_splunk
Splunk Employee
Splunk Employee

It looks as if these buckets that have errors are going back 10 months (November 27, 2018).
Can you check if the data is still in Splunk? If it is, can you check if these buckets made it to S3 despite of the exception?

0 Karma

sambhram
New Member

Yes the data are present in Splunk Indexer and it never make to S3 (the archive target)

0 Karma

rdagan_splunk
Splunk Employee
Splunk Employee

Can I assume that majority of the buckets made it to S3, but the above 5 did not?
Are you using S3A or S3 in the VIX? S3A does not have size limitation, so I am trying to eliminate that as the cause

0 Karma

sambhram
New Member

Nothing move to S3 at all. I am using S3A

0 Karma

rdagan_splunk
Splunk Employee
Splunk Employee

Let me know and we can try to debug these issues? My email is rdagan@splunk.com

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.