Deployment Architecture
Highlighted

Splunk Smartstore: Remote Storage Http Error Status Codes

Explorer

I have configured Smart Store on our Splunk cluster recently, after the changes,
I was seeing 403 and 404 errors in Remote Storage Http Error Status Codes.

Is anyone had the similar issue ?

Highlighted

Re: Splunk Smartstore: Remote Storage Http Error Status Codes

Explorer

Did you ever find a resolution to this issue? We are encountering the same thing.

0 Karma
Highlighted

Re: Splunk Smartstore: Remote Storage Http Error Status Codes

Explorer

No, we did not

0 Karma
Highlighted

Re: Splunk Smartstore: Remote Storage Http Error Status Codes

Explorer

posted in answer

0 Karma
Highlighted

Re: Splunk Smartstore: Remote Storage Http Error Status Codes

Path Finder

We are seeing the same thing in our environment. 403's doing REST.GET.BUCKETVERSIONS and 404's on REST.HEAD.OBJECT for keys ending in "receipts.json" or "done". Searches don't seem to be having problems, but we have a very high error count.

0 Karma
Highlighted

Re: Splunk Smartstore: Remote Storage Http Error Status Codes

Explorer

which version of Splunk are you on ?

0 Karma
Highlighted

Re: Splunk Smartstore: Remote Storage Http Error Status Codes

Path Finder

version 7.2.5.1

0 Karma
Highlighted

Re: Splunk Smartstore: Remote Storage Http Error Status Codes

Explorer

version 7.3.0

0 Karma
Highlighted

Re: Splunk Smartstore: Remote Storage Http Error Status Codes

Explorer

Based on our troubleshooting, Splunk is issuing a getobject of receipt.json (with the whole bucket details in the object name) before even performing a putobject and this should result in 404. As per Splunk support this can be ignored unless you see 404 for the same object multiple times.

0 Karma
Highlighted

Re: Splunk Smartstore: Remote Storage Http Error Status Codes

Explorer
{
    "Version": "***",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:GetLifecycleConfiguration",
                "s3:DeleteObjectVersion",
                "s3:ListBucketVersions",
                "s3:GetBucketLogging",
                "s3:RestoreObject",
                "s3:ListBucket",
                "s3:GetBucketVersioning",
                "s3:PutObject",
                "s3:GetObject",
                "s3:PutLifecycleConfiguration",
                "s3:GetBucketCORS",
                "s3:DeleteObject",
                "s3:GetBucketLocation",
                "s3:GetObjectVersion"
            ],
            "Resource": [
                "arn:aws:s3:::your-smartstore-bucket-name/*",
                "arn:aws:s3:::your-smartstore-bucket-name"
            ]
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets",
                "s3:HeadBucket"
            ],
            "Resource": "*"
        }
    ]
}

this bucket policy has zeroed my 403 errors , as far as 404 errors Splunk confirmed that this will be fixed in the future releases.

View solution in original post

0 Karma