Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
- About bendter
bendter
Explorer
Member since:
01-14-2015
06-09-2022
Community Statistics
| Posts | 9 |
| Solutions | 1 |
| Karma Given | 0 |
| Karma Received | 3 |
| Member Since | 01-14-2015 |
Activity Feed
- Got Karma for Re: Splunk Add-on for Amazon Web Services: Why are we unable to collect AWS cloudtrail data and getting and s3Bucket error?. 06-05-2020 12:47 AM
- Got Karma for Re: Splunk Add-on for Amazon Web Services: Why are we unable to collect AWS cloudtrail data and getting and s3Bucket error?. 06-05-2020 12:47 AM
- Got Karma for Re: Splunk Add-on for Amazon Web Services: Why are we unable to collect AWS cloudtrail data and getting and s3Bucket error?. 06-05-2020 12:47 AM
- Posted Re: How to configure Splunk to access and parse AWS GovCloud Cloudtrail logs? on All Apps and Add-ons. 01-24-2015 08:36 PM
- Posted Re: Splunk Add-on for Amazon Web Services: Why are we unable to collect AWS cloudtrail data and getting and s3Bucket error? on All Apps and Add-ons. 01-24-2015 01:49 PM
- Posted Re: Splunk Add-on for Amazon Web Services: Why are we unable to collect AWS cloudtrail data and getting and s3Bucket error? on All Apps and Add-ons. 01-24-2015 11:55 AM
- Posted Re: Splunk Add-on for Amazon Web Services: Why are we unable to collect AWS cloudtrail data and getting and s3Bucket error? on All Apps and Add-ons. 01-24-2015 11:54 AM
- Posted Re: Splunk Add-on for Amazon Web Services: Why are we unable to collect AWS cloudtrail data and getting and s3Bucket error? on All Apps and Add-ons. 01-15-2015 07:11 AM
- Posted Splunk Add-on for Amazon Web Services: Why are we unable to collect AWS cloudtrail data and getting and s3Bucket error? on All Apps and Add-ons. 01-14-2015 03:28 PM
- Tagged Splunk Add-on for Amazon Web Services: Why are we unable to collect AWS cloudtrail data and getting and s3Bucket error? on All Apps and Add-ons. 01-14-2015 03:28 PM
- Tagged Splunk Add-on for Amazon Web Services: Why are we unable to collect AWS cloudtrail data and getting and s3Bucket error? on All Apps and Add-ons. 01-14-2015 03:28 PM
Topics I've Started
| Subject | Karma | Author | Latest Post |
|---|---|---|---|
| 0 |
01-24-2015
08:36 PM
We are also trying to configure Splunk to gather cloudtrail logs from govcloud. It appears that it is not possible at this time. We have accounts in both the AWS general cloud and govcloud. Splunk gathers cloudtrail logs fine from the general cloud. But Splunk cannot gather cloudtrail logs from govcloud.
The issue appears to be that Splunk is attempting to log into into the general cloud for S3 and cannot find the S3 cloudtrail files for govcloud. All SQS data is correct and retrieved okay. But the aws_cloudtrail.py script fails to find the S3 file. You can see the error very easily by trying to configure a Splunk S3 input with the govcloud user:
Failed to fetch data: In handler 'splunk_ta_aws_s3buckets': Unexpected error "" from python handler: "S3ResponseError: 403 Forbidden InvalidAccessKeyId The AWS Access Key Id you provided does not exist in our records.AAAAAAAAAAAAAAAAAAAA8888888888888WWWWWWWWWWWWWW=". See splunkd.log for more details.
This same splunk AWS users works fine when configuring a cloudtrail input and can even find the correct SQS queue. It just fails when it trys to find the S3 file.
Does anyone know how to configure the Splunk AWS add-on to access S3 data in the govcloud region? It seems the Splunk AWS user should have a flag to set if this is a govcloud user.
... View more
01-24-2015
01:49 PM
2 Karma
Okay, I figured out the issue. It was operator error.
We had configured SNS on the cloudtrail bucket NOT on cloudtrail itself. Everything in AWS appeared working since we were getting message in the SQS queue as files were added from cloudtrail.
I removed the SNS notification from the S3 bucket and configured cloudtrail to post to the SNS. I also removed and recreated the SQS queue to get notifications from the new SNS topic.
Everything is now working as it should.
Thanks to everyone who looked into this for us.
... View more
01-24-2015
11:55 AM
I also added this statement to see what "keys" were available:
logger.log(logging.INFO, "**** keys: %s",message.keys())
The output is:
2015-01-24 11:06:11,299 INFO pid=9081 tid=MainThread
file=aws_cloudtrail.py:process_notifications:297 | **** keys: [u'Records']
It looks to me like the only key is "Records" and all other keys are invalid. But since I don't understand JSON I don't really have much of an idea as to why the code is failing.
In another post someone said to try this in the code:
#message = json.loads(envelope['Message'])
message = envelope
I tried that and it didn't work. It gave the same KeyError message. Based on the apps directory in the post it looks like that was based on an earlier version of the AWS splunk add-on.
Any help would be much appreciated!
... View more
01-24-2015
11:54 AM
1 Karma
I have further debugged this issue but I am still stumped. Please note I am NOT a JSON programmer.
The issue appearss to be caused by the the SQS message received by Splunk. Here is the line of code that generates the KeyError from aws_cloudtrail.py:
bucket_name = message['s3Bucket']
The key "S3Bucket" does not appear to exist. I dumped the SQS Messsage by adding this line to aws_cloudtrail.py:
logger.log(logging.INFO, "**** message: %s",json.dumps(message))
Here is the output (with some information obfuscated):
2015-01-24 11:06:11,299 INFO pid=9081 tid=MainThread file=aws_cloudtrail.py:process_notifications:295 | **** message: {"Records": [{"requestParameters": {"sourceIPAddress": "0.0.0.0"}, "userIdentity": {"principalId": "AWS:ARxxxxxxxxxxxxxxxxxxxx:i-69999999"}, "eventVersion": "2.0", "s3": {"bucket": {"ownerIdentity": {"principalId": "AAAAAAAAAAAAAA"}, "name": "cloudtrail", "arn": "arn:aws:s3:::cloudtrail"}, "s3SchemaVersion": "1.0", "object": {"eTag": "axxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", "size": 8450, "key": "Folder1/Folder2/Folder3/us-west-2/2015/01/21/688888888888_CloudTrail_us-west-2_20150121T1000Z_XXXXXXXXXXXXXX.json.gz"}, "configurationId": "Splunk"}, "eventSource": "aws:s3", "responseElements": {"x-amz-id-2": "+XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX=", "x-amz-request-id": "AAAAAAAAAAAAAAAA"}, "eventTime": "2015-01-21T10:00:03.710Z", "awsRegion": "us-west-2", "eventName": "ObjectCreated:Put"}]}
(continued in next comment)
... View more
01-15-2015
07:11 AM
The error message suggests a permission issue however I can log into AWS as the splunk user and download files from the cloudtrail S3 bucket manually. Originally I suspected our splunk credentials to AWS weren't working but I can use the same splunk AWS user to import data from cloudwatch.
... View more
01-14-2015
03:28 PM
Splunk Version 6.2.0
Splunk Add-on for AWS version 1.1.0
We have attempted to follow the installation instructions for the Splunk AWS Add-on. We have Cloudtrail enabled dumping to an S3 bucket. We have SNS notifications configured as well as the SQS queue. Messages are showing as available in the SQS queue. We have a Splunk user set up with full access to S3 buckets and Cloudtrail.
On the Splunk side we have the Cloudtrail and Cloudwatch inputs configured. We can successfully receive data from Cloudwatch.
We cannot import any data from Cloudtrail. Does anyone have any ideas what we might be missing?
Here are the log entries:
aws_cloudtrail.log:
2015-01-14 14:48:46,228 INFO pid=14944 tid=MainThread file=aws_cloudtrail.py::409 | STARTED:
2015-01-14 14:48:46,229 DEBUG pid=14944 tid=MainThread file=aws_cloudtrail.py:stream_events:178 | Start streaming.
2015-01-14 14:48:46,229 DEBUG pid=14944 tid=MainThread file=aws_cloudtrail.py:stream_events:196 | blacklist regex for eventNames is ^(?:Describe|List|Get)
2015-01-14 14:48:46,229 INFO pid=14944 tid=MainThread file=aws_cloudtrail.py:get_access_key_pwd_real:109 | get account name: Splunk General access user
2015-01-14 14:48:46,254 DEBUG pid=14944 tid=MainThread file=aws_cloudtrail.py:stream_events:210 | Connect to S3 & Sqs sucessfully
2015-01-14 14:48:46,563 CRITICAL pid=14944 tid=MainThread file=aws_cloudtrail.py:stream_events:278 | Outer catchall: KeyError: 's3Bucket'
2015-01-14 14:48:46,563 INFO pid=14944 tid=MainThread file=aws_cloudtrail.py::411 | EXITED: 1
splunkd.log:
01-14-2015 14:48:46.008 -0800 INFO ExecProcessor - New scheduled exec process: python /opt/splunk/etc/apps/Splunk_TA_aws/bin/aws_cloudwatch.py
01-14-2015 14:48:46.008 -0800 INFO ExecProcessor - interval: 30000 ms
01-14-2015 14:48:46.009 -0800 INFO ExecProcessor - New scheduled exec process: /opt/splunk/bin/splunkd instrument-resource-usage
01-14-2015 14:48:46.009 -0800 INFO ExecProcessor - interval: 0 ms
01-14-2015 14:48:46.598 -0800 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/aws_cloudtrail.py" ERRORs3Bucket
... View more
