Getting Data In

Using Custom option for ingesting VPC flow logs into Splunk

paleewawa
Explorer

Hello there, 

We are looking to use the Custom option to send vpc flow log data to Splunk Cloud. Previously we were using the default set of fields. There's a need to ingest additional fields without using the "all fields" option in order to save on data ingest. 
The issue appears to be with the Regex where the add-on cannot mix and match field names, rather it needs to be in a particular order otherwise the data is not parsed properly.

Default Format:
${version} ${account-id} ${interface-id} ${srcaddr} ${dstaddr} ${srcport} ${dstport} ${protocol} ${packets} ${bytes} ${start} ${end} ${action} ${log-status}

Custom Format
${version} ${account-id} ${vpc-id} ${subnet-id} ${interface-id} ${instance-id} ${flow-direction} ${srcaddr} ${dstaddr} ${srcport} ${dstport} ${pkt-srcaddr} ${pkt-dstaddr} ${protocol} ${packets} ${bytes} ${start} ${end} ${action} ${log-status}

This is preventing us from being able to include additional fields that can be useful to our team, without ingesting everything. Has anyone else encountered this problem before? 

Labels (1)
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Splunk Add-on for Microsoft Windows 11.0.3: Parse Once, Search Faster

The updated Splunk Add-on for Microsoft Windows helps customers search Windows events faster by reducing ...

What's New in Splunk Enterprise Security 8.8 and Beyond

Threats move at machine speed. Keeping up should not depend on having the biggest budget, the largest team, or ...

A Four-Part Event Series: State of Observability - Where AI Meets Performance

As AI agents take on more complex tasks, teams need greater visibility into how they behave, what they cost, ...