All Apps and Add-ons

What is the recommended way to ingest AWS Cloudwatch Logs?

New Member

According to the AWS add-on for Splunk, it is strongly recommended to avoid using the Cloudwatch Logs input due to deprecation.

Screenshot from the docs at :

Deprecation warning

I have countless AWS Lambda functions which dump their logs to CloudWatch Logs. What is the recommended way to ingest these? Ideally, I'd like them to be available in Splunk in real time, similarly to how they are with our server applications pushing logs through SplunkForwarder.

0 Karma

New Member

Has anyone else come up with a way to have Splunk pull from AWS versus push (which the later answers are using)? We have our Splunk on-prem and our AWS Cloudwatch logs (no VPC logs) and have a firewall inbetween. The best approach would be to do a pull (which is what we had hoped this plugin would do) but it sounds like that is not the case (or not reliable)? Has anyone else accomplished what we are after or are we stuck with the push option only?

0 Karma


I believe the Amazon Kensis firehose is the new preferred method according to this Splunk blog post

The application for Splunk Add-on for Amazon Kinesis Firehose appears to be the relevant application...

EDIT: not available in all regions such as Australia.... might be the alternative for now as per apringle's comment.

0 Karma


I'm currently investigating how to do this myself. It seems like the referenced docs have been updated to show that VPC Flow Logs specifically should be avoided, and no longer says CloudWatch Logs in general. I don't know for certain though (I haven't seen the official deprecation notice). Here's the current text:

Splunk strongly recommends against using the CloudWatch Logs inputs to collect VPC Flow Logs data (source type: aws:cloudwatchlogs:vpcflow) since the input type will be deprecated in upcoming releases. Configure Kinesis inputs to collect VPC Flow Logs instead. The add-on includes index-time logic to perform the correct knowledge extraction for these events through the Kinesis input as well.

However, it seems like this is probably a good alternative solution to get any CloudWatch Logs into Splunk:

My understanding is that it would give you a real-time push rather than the polling method that the AWS add-on input provides.

0 Karma

New Member

Does anyone have a answer of how to build a lambda function through python (snippet if anyone has it) to ingest cloudwatch logs to be available in real time to splunk,

0 Karma

New Member
0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Get More Out of Your Security Practice With a SIEM

Get More Out of Your Security Practice With a SIEMWednesday, July 31, 2024  |  11AM PT / 2PM ETREGISTER ...