Splunk Enterprise

Ingesting data into Splunk enterprise from 3rd party aws s3 bucket?



We have a requirement to pull data from third-party aws account. Third party provider will push the data to a S3 bucket in their aws account and we are looking to pull that to an on-prem Splunk instance. There is an aws Splunk add-in splunkbase , are we able to use this add-on to pull data from a third-party aws account , if so how is it authenticated against third-party account? Please point me to any documentation available 

Any suggestions?

Labels (1)
0 Karma


hi @Luckyani 

You will be using the Splunk Add-on for AWS (Splunk Add-on for Amazon Web Services (AWS) | Splunkbase)
You'll need to create and configure an IAM role in the add-on input, that has permission to read the S3 resources hosted by the third party. 
Depending on the trust relationship between your on-prem and third party resources, you'll have to allow the traffic from their AWS infra to yours ~ whitelisting traffic. This needs to be done on both sides.  Ensure the IAM role specifies the trust relationship and is enabled to use correct permissions. Would strongly recommend avoiding overly permissive permissions, restrict on resources and limit actions.

Once you configure the account and specify the role correctly in the add-on, you should be able to see the details of resources  accessible from the third party AWS instance.

Configuring account in add-on
Manage accounts for the Splunk Add-on for AWS - Splunk Documentation

I would strongly recommend using SQS-based S3 input configuration for S3 data sources. This scales well for high frequency changes of datasets inS3.
Configure SQS-based S3 inputs for the Splunk Add-on for AWS - Splunk Documentation

 If  the reply helps, a karma upvote would be appreciated.

0 Karma
Get Updates on the Splunk Community!

Build Scalable Security While Moving to Cloud - Guide From Clayton Homes

 Clayton Homes faced the increased challenge of strengthening their security posture as they went through ...

Mission Control | Explore the latest release of Splunk Mission Control (2.3)

We’re happy to announce the release of Mission Control 2.3 which includes several new and exciting features ...

Cloud Platform | Migrating your Splunk Cloud deployment to Python 3.7

Python 2.7, the last release of Python 2, reached End of Life back on January 1, 2020. As part of our larger ...