Splunk Enterprise

Ingesting data into Splunk enterprise from 3rd party aws s3 bucket?

Luckyani
Explorer

Hi 

We have a requirement to pull data from third-party aws account. Third party provider will push the data to a S3 bucket in their aws account and we are looking to pull that to an on-prem Splunk instance. There is an aws Splunk add-in splunkbase , are we able to use this add-on to pull data from a third-party aws account , if so how is it authenticated against third-party account? Please point me to any documentation available 

Any suggestions?

Labels (1)
0 Karma

Gr0und_Z3r0
Contributor

hi @Luckyani 

You will be using the Splunk Add-on for AWS (Splunk Add-on for Amazon Web Services (AWS) | Splunkbase)
You'll need to create and configure an IAM role in the add-on input, that has permission to read the S3 resources hosted by the third party. 
Depending on the trust relationship between your on-prem and third party resources, you'll have to allow the traffic from their AWS infra to yours ~ whitelisting traffic. This needs to be done on both sides.  Ensure the IAM role specifies the trust relationship and is enabled to use correct permissions. Would strongly recommend avoiding overly permissive permissions, restrict on resources and limit actions.

Once you configure the account and specify the role correctly in the add-on, you should be able to see the details of resources  accessible from the third party AWS instance.

Configuring account in add-on
Manage accounts for the Splunk Add-on for AWS - Splunk Documentation

I would strongly recommend using SQS-based S3 input configuration for S3 data sources. This scales well for high frequency changes of datasets inS3.
Configure SQS-based S3 inputs for the Splunk Add-on for AWS - Splunk Documentation


 If  the reply helps, a karma upvote would be appreciated.

0 Karma
Get Updates on the Splunk Community!

2024 Splunk Career Impact Survey | Earn a $20 gift card for participating!

Hear ye, hear ye! The time has come again for Splunk's annual Career Impact Survey!  We need your help by ...

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...