I am storing data (JSON/CSV) in s3 bucket in AWS and I want to send this data into Splunk and data is updated every 5 minutes so I want to update or create a new data log in Splunk in every 5 minutes.
I am now trying by using Splunk add-on for AWS app but I don't know if it will help to send data inside the s3 bucket or not?
Can anyone tell me the right method or way to do it??
You can use splunk add-on for aws to send data from aws s3 to splunk. you should use SQS based S3 approach.
If the data volume is high, you can use this route s3 -> kinesis firehose -> Splunk (using HEC)
hope this helps.
Thank you for your reply.
one thing I want is that: every 5 minutes I want the data in that s3 file get send to Splunk. so, can we plan the timing as well??
and for HEC:
I tried using HEC previously but due to the error, I couldn't solve it.