All Apps and Add-ons

Splunk App for AWS: How to get Cloudwatch (vpc flow) logs into Splunk?

Explorer

Hi

I am using Splunk in AWS and, using the the Splunk App for AWS, want to get VPC Flow logs into Splunk. VPC Flow logs are put into Cloudwatch Logs. Does anyone know how to get Cloudwatch logs into Splunk?

Thanks

Nick

0 Karma
1 Solution

Splunk Employee
Splunk Employee

The Splunk Add-on for AWS version 2.0.0 includes support for ingesting your VPC Flow Logs data. Get it here: https://splunkbase.splunk.com/app/1876/

There is also a new version of the Splunk App for AWS, now officially Splunk-supported, that provides dashboards for that data. http://splunkbase.splunk.com/app/1274/

View solution in original post

Splunk Employee
Splunk Employee

The Splunk Add-on for AWS version 2.0.0 includes support for ingesting your VPC Flow Logs data. Get it here: https://splunkbase.splunk.com/app/1876/

There is also a new version of the Splunk App for AWS, now officially Splunk-supported, that provides dashboards for that data. http://splunkbase.splunk.com/app/1274/

View solution in original post

New Member

Hi Everyone -

I just ran across this project this morning. It has connectors for CWL to S3 or Elasticsearch out of the box, but it shouldn't be too difficult to forge a connector for Splunk.

https://github.com/awslabs/cloudwatch-logs-subscription-consumer

Hope it helps!

jp

0 Karma

Based on this blog posting from Splunk, it sounds like VPC flow logs are something they are working to add.

http://blogs.splunk.com/2015/08/04/an-aws-summer-part-1/

If there's a way to do it now, that would be great as I'm looking to do the same.

0 Karma

Splunk Employee
Splunk Employee

You'll want to install the Splunk Add-on for Amazon.
Have you checked the docs? - http://docs.splunk.com/Documentation/AddOns/latest/AWS/ConfigureInputs

1) You'll need to grant permission from within AWS to the account the Splunk is using to connect into AWS with.
2) You'll need to configure CloudWatch inputs as referenced in the doc above.

Hope this helps.

Explorer

Hi

Thanks for your response. I have installed the Splunk Add-On for AWS.

I can see from the docs link that you posted how to capture a CloudWatch Metric, but not how to capture a CloudWatch Log. This should involve getting Splunk to read from the CloudWatch Log stream to which events are written - this is different from reading published metrics.

Thanks

Nick

0 Karma

Splunk Employee
Splunk Employee

this might be useful--a kind twitter user posted it in response to your question: https://github.com/awslabs/cloudwatch-logs-subscription-consumer
(see https://twitter.com/fnordpig/status/634766161394167808 )