Getting Data In
Highlighted

Splunk: Archive to S3 or S3 compatible Object Store using Hadoop ?

New Member

I am seeing the following error message while trying to archive to S3. The logs are from "splunk_archiver.log". Any pointer as how to fix this ?

2019-09-09 06:09:11.127 -0700 ERROR Roller - Exception when deciding if bucket should be copied. bucketname=db1566816265154337075753D55DFE1B-5E59-4A40-A430-7A8334A5FE47, splunkindex=main, dir=/opt/splunk/var/lib/splunk/defaultdb/db/db1566816265154337075753D55DFE1B-5E59-4A40-A430-7A8334A5FE47
2019-09-09 06:20:54.669 -0700 ERROR Roller - Exception when deciding if bucket should be copied. bucketname=db1553167044154541844734D55DFE1B-5E59-4A40-A430-7A8334A5FE47, splunkindex=main, dir=/opt/splunk/var/lib/splunk/defaultdb/db/db1553167044154541844734D55DFE1B-5E59-4A40-A430-7A8334A5FE47
2019-09-09 06:33:11.420 -0700 ERROR Roller - Exception when deciding if bucket should be copied. bucketname=db1554020317154629262898D55DFE1B-5E59-4A40-A430-7A8334A5FE47, splunkindex=main, dir=/opt/splunk/var/lib/splunk/defaultdb/db/db1554020317154629262898D55DFE1B-5E59-4A40-A430-7A8334A5FE47
2019-09-09 06:44:50.075 -0700 ERROR Roller - Exception when deciding if bucket should be copied. bucketname=db1549687163154634372993D55DFE1B-5E59-4A40-A430-7A8334A5FE47, splunkindex=main, dir=/opt/splunk/var/lib/splunk/defaultdb/db/db1549687163154634372993D55DFE1B-5E59-4A40-A430-7A8334A5FE47
2019-09-09 06:57:14.182 -0700 ERROR Roller - Exception when deciding if bucket should be copied. bucketname=db154895476115463466431D55DFE1B-5E59-4A40-A430-7A8334A5FE47, splunkindex=main, dir=/opt/splunk/var/lib/splunk/defaultdb/db/db154895476115463466431D55DFE1B-5E59-4A40-A430-7A8334A5FE47

0 Karma
Highlighted

Re: Splunk: Archive to S3 or S3 compatible Object Store using Hadoop ?

Splunk Employee
Splunk Employee

It looks as if these buckets that have errors are going back 10 months (November 27, 2018).
Can you check if the data is still in Splunk? If it is, can you check if these buckets made it to S3 despite of the exception?

0 Karma
Highlighted

Re: Splunk: Archive to S3 or S3 compatible Object Store using Hadoop ?

New Member

Yes the data are present in Splunk Indexer and it never make to S3 (the archive target)

0 Karma
Highlighted

Re: Splunk: Archive to S3 or S3 compatible Object Store using Hadoop ?

Splunk Employee
Splunk Employee

Can I assume that majority of the buckets made it to S3, but the above 5 did not?
Are you using S3A or S3 in the VIX? S3A does not have size limitation, so I am trying to eliminate that as the cause

0 Karma
Highlighted

Re: Splunk: Archive to S3 or S3 compatible Object Store using Hadoop ?

New Member

Nothing move to S3 at all. I am using S3A

0 Karma
Highlighted

Re: Splunk: Archive to S3 or S3 compatible Object Store using Hadoop ?

Splunk Employee
Splunk Employee

Let me know and we can try to debug these issues? My email is rdagan@splunk.com

0 Karma