Splunk Enterprise

Ingesting data into Splunk enterprise from 3rd party aws s3 bucket?

Luckyani
Explorer

Hi 

We have a requirement to pull data from third-party aws account. Third party provider will push the data to a S3 bucket in their aws account and we are looking to pull that to an on-prem Splunk instance. There is an aws Splunk add-in splunkbase , are we able to use this add-on to pull data from a third-party aws account , if so how is it authenticated against third-party account? Please point me to any documentation available 

Any suggestions?

Labels (1)
0 Karma

Gr0und_Z3r0
Contributor

hi @Luckyani 

You will be using the Splunk Add-on for AWS (Splunk Add-on for Amazon Web Services (AWS) | Splunkbase)
You'll need to create and configure an IAM role in the add-on input, that has permission to read the S3 resources hosted by the third party. 
Depending on the trust relationship between your on-prem and third party resources, you'll have to allow the traffic from their AWS infra to yours ~ whitelisting traffic. This needs to be done on both sides.  Ensure the IAM role specifies the trust relationship and is enabled to use correct permissions. Would strongly recommend avoiding overly permissive permissions, restrict on resources and limit actions.

Once you configure the account and specify the role correctly in the add-on, you should be able to see the details of resources  accessible from the third party AWS instance.

Configuring account in add-on
Manage accounts for the Splunk Add-on for AWS - Splunk Documentation

I would strongly recommend using SQS-based S3 input configuration for S3 data sources. This scales well for high frequency changes of datasets inS3.
Configure SQS-based S3 inputs for the Splunk Add-on for AWS - Splunk Documentation


 If  the reply helps, a karma upvote would be appreciated.

0 Karma
Get Updates on the Splunk Community!

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...

SignalFlow: What? Why? How?

What is SignalFlow? Splunk Observability Cloud’s analytics engine, SignalFlow, opens up a world of in-depth ...

Federated Search for Amazon S3 | Key Use Cases to Streamline Compliance Workflows

Modern business operations are supported by data compliance. As regulations evolve, organizations must ...