All Apps and Add-ons

Can we integrate the Splunk App for AWS in our Splunk Enterprise environment?

aaguirr1
New Member

Hi all,

We have deployed Splunk Enterprise on premise to manage the logs of different IT assets. At present, we are going to use AWS services. (EC2, VPC, RDS, S3, CloudConfig, etc.)

I have the next question:

1) Can we integrate Splunk App for AWS in our Splunk Enterprise? I would like to have the same dashboard to manage the logs of AWS console and IT assets.

2) We have 3 accounts of AWS (Prod, Dev and R&D). Do we need install an indexer in each account? Overall 3 indexers.

If you need more information, please let me know.

Thanks in advance.

Regards,
Arsenio

0 Karma

Jeremiah
Motivator

You can deploy the Splunk app for AWS in a several different ways. You should think about the app as having 3 functions: data collection, storage, and UI. You can run all or some of those functions in your data center, in AWS or in a hybrid deployment.

For example, you could run an entirely on-prem solution, with a search head that runs the AWS app (provides the UI capability), but also collects data from AWS (data collection), and forwards that data to an indexer (the storage). Or you could deploy a search head in your corporate data center with the AWS app installed, and then perform the collection on a forwarder running the AWS app in AWS and send data to an indexer also running in AWS.

You might want an indexer in each account (VPC) if you are running forwarders on your EC2 instances. If not, there isn't any direct connection between your account/VPCs and the Splunk app for AWS. The app just needs connectivity to AWS endpoints which are accessible over the internet.

0 Karma

n6BXGybt
Path Finder

I'm not sure about question 1) however for question 2) I suspect that you don't need an indexer for the AWS specific logs as there are no Sizing and Performance considerations.

Reference: http://docs.splunk.com/Documentation/AWS/4.2.1/Installation/Sizingandcost

In my case, I have four AWS accounts (DEV, QA, UAT, and PROD). Since assume role is not supported, I have an IAM user account in each AWS account - DEV-SPLUNKAPP-USER for example. This user has in inline policy configured as below and per the Splunk documentation for using one policy for all inputs:

Reference: http://docs.splunk.com/Documentation/AWS/4.2.1/Installation/ConfigureyourAWSpermissions

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "sqs:GetQueueAttributes",
                "sqs:ListQueues",
                "sqs:ReceiveMessage",
                "sqs:GetQueueUrl",
                "sqs:SendMessage",
                "sqs:DeleteMessage",
                "s3:ListBucket",
                "s3:GetObject",
                "s3:GetBucketLocation",
                "s3:ListAllMyBuckets",
                "config:DeliverConfigSnapshot",
                "config:DescribeConfigRules",
                "config:DescribeConfigRuleEvaluationStatus",
                "config:GetComplianceDetailsByConfigRule",
                "config:GetComplianceSummaryByConfigRule",
                "config:DescribeDeliveryChannels",
                "cloudtrail:DescribeTrails",
                "autoscaling:Describe*",
                "cloudwatch:Describe*",
                "cloudwatch:Get*",
                "cloudwatch:List*",
                "sns:Get*",
                "sns:List*",
                "logs:DescribeLogGroups",
                "logs:DescribeLogStreams",
                "logs:GetLogEvents",
                "ec2:DescribeInstances",
                "ec2:DescribeReservedInstances",
                "ec2:DescribeSnapshots",
                "ec2:DescribeRegions",
                "ec2:DescribeKeyPairs",
                "ec2:DescribeNetworkAcls",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeSubnets",
                "ec2:DescribeVolumes",
                "ec2:DescribeVpcs",
                "rds:DescribeDBInstances",
                "cloudfront:ListDistributions",
                "elasticloadbalancing:DescribeLoadBalancers",
                "elasticloadbalancing:DescribeInstanceHealth",
                "inspector:Describe*",
                "inspector:List*"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

In my case I also created a separate index in Splunk (Settings --> Indexes) for each AWS service. (eg. config_logs, cloudwatch_logs, etc.)

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...