Getting Data In

Getting Splunk to correctly read custom AWS VPC flow logs

olawalePS
Path Finder

Hello All,

I recently started ingesting vac flow logs from my AWS environment using the data manager app, and everything works fine in terms of getting the logs into splunk. 

There is however one issue, when creating the VPC flow logs on AWS, we opted for a custom format to be able to glean additional fields like the "pkt-srcaddr" and pat-dstaddr". As a result of this, Splunk does not correctly interpret the logs on the console. I believe that Splunk is reading the logs using the default log format detailed below:

Default Format:
${version} ${account-id} ${interface-id} ${srcaddr} ${dstaddr} ${srcport} ${dstport} ${protocol} ${packets} ${bytes} ${start} ${end} ${action} ${log-status}


how do I get it to read the logs using the custom format detailed below:

Custom Format
${version} ${account-id} ${vpc-id} ${subnet-id} ${interface-id} ${instance-id} ${flow-direction} ${srcaddr} ${dstaddr} ${srcport} ${dstport} ${pkt-srcaddr} ${pkt-dstaddr} ${protocol} ${packets} ${bytes} ${start} ${end} ${action} ${log-status}

Labels (2)
Tags (2)
0 Karma
Get Updates on the Splunk Community!

Splunk Mobile: Your Brand-New Home Screen

Meet Your New Mobile Hub  Hello Splunk Community!  Staying connected to your data—no matter where you are—is ...

Introducing Value Insights (Beta): Understand the Business Impact your organization ...

Real progress on your strategic priorities starts with knowing the business outcomes your teams are delivering ...

Enterprise Security (ES) Essentials 8.3 is Now GA — Smarter Detections, Faster ...

As of today, Enterprise Security (ES) Essentials 8.3 is now generally available, helping SOC teams simplify ...