All Apps and Add-ons

Splunk Add on for AWS

mayur81
New Member

Hey there,

I am planning to use Splunk Add on for AWS to monitor AWS using on Premise Splunk installation. I have few questions related to this.

  • What is the best way to connect On Premise Splunk to AWS for getting the AWS data?
  • Since the add on will be using AWS API/services for getting AWS data, how can I find out this cost? There is no documentation on how the Add On uses AWS API/services.
Tags (2)
0 Karma

amiracle
Splunk Employee
Splunk Employee

First, check out this white paper we put together to walk you through ways to collect data from AWS and into Splunk : https://www.splunk.com/en_us/form/getting-data-into-gdi-splunk-from-aws.html . Next, if I were to set this up I would use a heavy forwarder in AWS with the Splunk TA for AWS installed then use an EC2 IAM Role to authenticate and collect the data. The HF can then forward the data to your on-prem Splunk using a standard forwarding port (TCP:9997).

For higher volume data, I would recommend using Kinesis Data Firehose (KDF) or Lambda functions and send the data via HEC to your Splunk deployment. This will require a public facing IP (for KDF) and a properly signed SSL Cert. You might need to either setup a HF tier or send directly to your indexers with a load balancer as the HEC endpoint.

Cost : The cost for collecting the data is going to come down to a few factors. First, is the amount of data being sent over the Internet Gateway and back to your data center. You can calculate the costs using the AWS calculator :https://calculator.s3.amazonaws.com/index.html . Next is the amount of times an API is hit from Splunk to collect the events. The average cost again can be set on the inputs.conf (or the inputs in the UI). The API calls are usually low, but the amount of data and the method of collecting the data will have an impact on the overall cost of collecting data.

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Best Practices: Splunk auto adjust pipeline queue

When you enable autoAdjustQueue in Splunk, maxSize should be understood as the queue size Splunk starts with ...

Request for Professional Development: Attending .conf26

Winning Over the Boss: Your Pass to .conf26 conf26 is going to be here before you know it. If don't already ...