Getting Data In

How to send data (JSON/CSV) from AWS s3 to Splunk at 5 minute intervals?

rsilwal7
Loves-to-Learn Lots

Hello,

I am storing data (JSON/CSV) in s3 bucket in AWS and I want to send this data into Splunk and data is updated every 5 minutes so I want to update or create a new data log in Splunk in every 5 minutes.

I am now trying by using Splunk add-on for AWS app but I don't know if it will help to send data inside the s3 bucket or not?

Can anyone tell me the right method or way to do it??

Thank you!

Labels (5)
0 Karma

anilchaithu
Builder

@rsilwal7 

You can use splunk add-on for aws to send data from aws s3 to splunk. you should use SQS based S3 approach.

If the data volume is high, you can use this route s3 -> kinesis firehose -> Splunk (using HEC)

https://docs.splunk.com/Documentation/AddOns/released/AWS/SQS-basedS3

hope this helps.

0 Karma

rsilwal7
Loves-to-Learn Lots

Thank you for your reply.

one thing I want is that: every 5 minutes I want the data in that s3 file get send to Splunk. so, can we plan the timing as well?? 

and for HEC:

I tried using HEC previously but due to the error, I couldn't solve it.

 

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk Observability Cloud – June 2025

What’s New in Splunk Observability Cloud – June 2025 We are excited to announce the latest enhancements to ...

Almost Too Eventful Assurance: Part 2

Work While You SleepBefore you can rely on any autonomous remediation measures, you need to close the loop ...

Leveraging Detections from the Splunk Threat Research Team & Cisco Talos

 Stay ahead of today’s evolving threats with the combined power of the Splunk Threat Research Team (STRT) and ...