Hi
We are trying out the Splunk App and Add-on for AWS for first time and this is my first time on this forum.
The Add-on does make the connection OK and provides in the GUI drop-down a list of valid AWS queues. After selecting the appropriate queue, the following error appears. Any advice / thoughts on next steps please?
2016-01-28 11:21:11,137 ERROR pid=14264 tid=MainThread
file=aws_cloudtrail.py:process_CT_notifications:594 | S3ResponseError:
400 Bad Request: InvalidArgument - Requests specifying Server Side Encryption with AWS KMS managed keys require AWS Signature Version 4.:
Thanks in advance
Could you please add the following entry to splunk-launch.conf and restart splunkd
S3_USE_SIGV4 = True
This eliminated the error for me - thanks!
You have encryption enabled on your Cloudtrail logs.
The boto library that the Splunk add on uses does not pass the correct version of the AWS signature required by SSE-enabled S3 buckets by default:
https://forums.aws.amazon.com/thread.jspa?threadID=165286
You can, however force boto to use the correct version of the signature, see the section titled "Specifying Signature Version in Request Authentication" for Python boto sdk.
http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingAWSSDK.html
You'll need to add the following line to the boto config file.
[s3] use-sigv4 = True
The doc below lists your options for the boto.cfg file. I'd suggest either /etc/boto.cfg or the .boto file in the home directory of your Splunk user (the account you run splunk as).
http://boto.cloudhackers.com/en/latest/boto_config_tut.html
/etc/boto.cfg - for site-wide settings that all users on this machine will use
(if profile is given) ~/.aws/credentials - for credentials shared between SDKs
(if profile is given) ~/.boto - for user-specific settings
~/.aws/credentials - for credentials shared between SDKs
~/.boto - for user-specific settings
Thanks for response, have tried it and it has got me further forward.
However have run into another issue (S3ResponseError: 400 Bad Request: None - 🙂 which I see others have experienced, but the resolution is unclear at this stage and/or could be with AWS possibly
e.g. https://answers.splunk.com/answers/207237/problem-fetching-logs-from-aws-s3-buckets.html
Hi Again
I'm going through the end-to-end setup with an AWS consultant to see how far we can progress it. At this stage we are finding the configuration of the AWS Add-On itself a bit of a dark art e.g. a current lack of clarity around Proxy configuration within the AWS Add-On / App. We'll pursue this a little further ourselves for now.
Thanks. R
Do you have any additional details from the error message? Make sure the AWS account you are using also has IAM permissions to access the KMS key.
Hi Again.
At the moment I'm work through the configuration of AWS APP and AWS-addon with support from a AWS consultant. Getting this add-on working a is feeling like a dark art. There seem to be a number of odd things going on e.g. exactly how and where its needs to be configured to use a Proxy. The seem to be multiple options (the UI and a variety of *.conf files). We'll take it as far as we can and then perhaps post another fresh query if required.
Thanks
R