Here are the latest instructions on how to deploy the Splunk App for AWS version 4.0 (https://splunkbase.splunk.com/app/1274/#/overview) in a distributed Splunk environment.
Installing AWS 2.0 app on A UF (distributed deployment)
Enable notifications of your logs to an SNS topic
You should see your Cloudtrail Topic which was create from previous set.
Capture the ARN, you will need this later, when you make a subscription.
Create a Topic:
1. Create an SQS queue to be used to watch for SNS notification of CloudTrail logs (i.e. cloudtrail2splunk above) 2. Goto the SQS Console and and make a new Queue:
Enable the ARN field and capture it.
Mine is: arn:aws:sqs:us-east-1:blah:splunk2cloudtrail
Goto Queue Action and add Permission as shown below:
To receive messages published to a topic, you have to subscribe an endpoint to that topic. An endpoint is a web server, an email address, or an Amazon SQS queue that can receive notification messages from Amazon SNS. Once you subscribe an endpoint to a topic and the subscription is confirmed, the endpoint will receive all messages published to that topic.
In this section you subscribe an endpoint to the topic you just created in the previous section. You configure the subscription to send the topic messages to your email account.
To subscribe to a topic
1. In the AWS Management Console, click My Subscriptions in the Navigation pane.
The My Subscriptions page opens.
2. Click the Create New Subscription button. The Subscribe dialog box appears:
Note that a single queue in any region may be subscribed to multiple topics from many regions,
if that configuration is desirable.
Installing on a Splunk Forwarder:
1. Copy the app to /tmp 2. As Splunk user untar it: [splunk@ernie tmp]$ tar zxf splunk-app-for-aws_20.tgz 3. Move the app to $SPLUNK_HOME/etc/apps [splunk@ernie tmp]$ mv SplunkAppforAWS ~splunk/etc/apps/ 4. Under "Settings, Data Inputs", create a new AWS CloudTrail Log input. Note if installing on a forwarder skip to step xxx. 5. Enter your AWS ID, Secret key, the region of your SQS queue, and the queue name
Access Key: blah
Secret Access Key: blah
6. For Forwarder config, create an inputs.conf file in etc/apps/SplunkAppforAWS/local: $ more inputs.conf [aws-cloudtrail://SplunkCloudTrail] exclude_describe_events = 1 host = ernie_forwarder index = aws-cloudtrail interval = 1 key_id = <ACCOUNT_KEY> remove_files_when_done = 0 secret_key = <SECRET-KEY> sqs_queue = splunk2cloudtrail sqs_queue_region = us-east-1 #_tzhint=GMT 7. (Optional) You can put the billing info in there if you want. Note you'll have to setup Central billing first but the file in local too (see other document): $ vi aws.conf # all this three stanzas are required, in order to run AWS App. [keys] # At least one entry required for this stanza # Format : # <accountno> = <company/group name without space> <aws-access-key> <aws-secret-key> <monthly spend limit for this account> #1122334455 = mycompany-name AAAAAAAAAAAAAAAAAAAA +++++BBBBBBBBBBBBBBBBBB/BBBB 10000 6162xxxxxxxx = freesoft <ACCESSKEY> <SECRET-KEY> 20 431xxxxxxxxx = splunk SECRETBLAH <SECRET-KEY> 20 [regions] # At least one entry required for this stanza rgn1 = eu-west-1 rgn2 = sa-east-1 rgn3 = us-east-1 rgn4 = ap-northeast-1 rgn5 = us-west-2 rgn6 = us-west-1 rgn7 = ap-southeast-1 rgn8 = ap-southeast-2 [misc] # Format : # corpkey = <company name/corp account name without space> <aws-access-key> <aws-secret-key> corpkey = freesoft <ACCESS-KEY> <SECRET-KEY> # acno is corp account number for AWS acno = 616xxxxxxxxx # s3bucket is bucket name where AWS bill csv files will be dropped s3bucket = aws_app
I am in a similar situation. Deploying the app via the master to the indexers does not appear to work due to hard-coded paths inside the app. The searches in the billing section also appear to use a local file (that would not be generated on the search heads if the input scripts are running on the indexer/s).