Getting Data In

Splunk: Archive to S3 or S3 compatible Object Store using Hadoop ?

sambhram
New Member

I am seeing the following error message while trying to archive to S3. The logs are from "splunk_archiver.log". Any pointer as how to fix this ?

2019-09-09 06:09:11.127 -0700 ERROR Roller - Exception when deciding if bucket should be copied. bucket_name=db_1566816265_1543370757_53_D55DFE1B-5E59-4A40-A430-7A8334A5FE47, splunk_index=main, dir=/opt/splunk/var/lib/splunk/defaultdb/db/db_1566816265_1543370757_53_D55DFE1B-5E59-4A40-A430-7A8334A5FE47
2019-09-09 06:20:54.669 -0700 ERROR Roller - Exception when deciding if bucket should be copied. bucket_name=db_1553167044_1545418447_34_D55DFE1B-5E59-4A40-A430-7A8334A5FE47, splunk_index=main, dir=/opt/splunk/var/lib/splunk/defaultdb/db/db_1553167044_1545418447_34_D55DFE1B-5E59-4A40-A430-7A8334A5FE47
2019-09-09 06:33:11.420 -0700 ERROR Roller - Exception when deciding if bucket should be copied. bucket_name=db_1554020317_1546292628_98_D55DFE1B-5E59-4A40-A430-7A8334A5FE47, splunk_index=main, dir=/opt/splunk/var/lib/splunk/defaultdb/db/db_1554020317_1546292628_98_D55DFE1B-5E59-4A40-A430-7A8334A5FE47
2019-09-09 06:44:50.075 -0700 ERROR Roller - Exception when deciding if bucket should be copied. bucket_name=db_1549687163_1546343729_93_D55DFE1B-5E59-4A40-A430-7A8334A5FE47, splunk_index=main, dir=/opt/splunk/var/lib/splunk/defaultdb/db/db_1549687163_1546343729_93_D55DFE1B-5E59-4A40-A430-7A8334A5FE47
2019-09-09 06:57:14.182 -0700 ERROR Roller - Exception when deciding if bucket should be copied. bucket_name=db_1548954761_1546346643_1_D55DFE1B-5E59-4A40-A430-7A8334A5FE47, splunk_index=main, dir=/opt/splunk/var/lib/splunk/defaultdb/db/db_1548954761_1546346643_1_D55DFE1B-5E59-4A40-A430-7A8334A5FE47

0 Karma

rdagan_splunk
Splunk Employee
Splunk Employee

It looks as if these buckets that have errors are going back 10 months (November 27, 2018).
Can you check if the data is still in Splunk? If it is, can you check if these buckets made it to S3 despite of the exception?

0 Karma

sambhram
New Member

Yes the data are present in Splunk Indexer and it never make to S3 (the archive target)

0 Karma

rdagan_splunk
Splunk Employee
Splunk Employee

Can I assume that majority of the buckets made it to S3, but the above 5 did not?
Are you using S3A or S3 in the VIX? S3A does not have size limitation, so I am trying to eliminate that as the cause

0 Karma

sambhram
New Member

Nothing move to S3 at all. I am using S3A

0 Karma

rdagan_splunk
Splunk Employee
Splunk Employee

Let me know and we can try to debug these issues? My email is rdagan@splunk.com

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...