All Apps and Add-ons

Splunk addon for AWS

bbiswabhusan
Explorer

Hello experts, im in the process of onboarding microservices on AWS lambda to splunk. the microservices are logging to cloudwatch at the moment. I have two options currently, using kinesis firehose lambda blueprint and splunk add-on for AWS. Just ant to know what are the specific cases where kinesis firehose must be used as I can see the add-on can process all kins of logs at the moment.
Can someone please guide.
Thanks
Biswa

0 Karma

amiracle
Splunk Employee
Splunk Employee

We published a white paper to help with this type of question: https://www.splunk.com/en_us/form/getting-data-into-gdi-splunk-from-aws.html .

The TL;DR version of the document for your specific use case : For high volume data, Kinesis Data Firehose (KDF) is one of the better options. If you require validation that the event made it into Splunk and you do not want to setup a HWF, then KDF makes the most sense.

For either lower volume data or data which does not require event validation, but you want to collect it serverlessly, then Lambda Functions make sense.

One last point, is that KDF can also be a less expensive path when compared to using CloudWatch Logs (https://aws.amazon.com/cloudwatch/pricing/ ) vs. (https://aws.amazon.com/kinesis/data-firehose/pricing/ ).

0 Karma

bbiswabhusan
Explorer

Hi @amiracle , thanks for your inputs.all the microservices are logging to cloudwatch at the moment. so we are considering using the splunk add-on for aws as we dont have vpc logs in scope at the moment. Can we go ahead, what is your thoughts on this.
Thanks

0 Karma

amiracle
Splunk Employee
Splunk Employee

Yes you can collect the CloudWatch Logs this way, but I would use either a Lambda function to collect them or Kinesis streams to avoid hitting your API limits by pulling the data via the Splunk Add-on.

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Get More Out of Your Security Practice With a SIEM

Get More Out of Your Security Practice With a SIEMWednesday, July 31, 2024  |  11AM PT / 2PM ETREGISTER ...