Hi Splunk Community,
I’ve set up Azure Firewall logging, selecting all firewall logs and archiving them to a storage account (Event Hub was avoided due to cost concerns). The configuration steps taken are as follows:
Log Archival:
- All Azure Firewall logs are set to archive in a storage account
Microsoft Cloud Add-On
- I added the storage account to the Microsoft Cloud Add-On using the secret key with the following permissions:
| Input/Action | API | Permissions | Role (IAM) | Default Sourcetype(s) / Sources |
| Azure Storage Table Azure Storage Blob | N/A | Access key OR Shared Access Signature: - Allowed services: Blob, Table - Allowed resource types: Service, Container, Object - Allowed permissions: Read, List | N/A | mscs:storage:blob ✅ (Received this) mscs:storage:blob:json ❌ mscs:storage:blob:xml❌ mscs:storage:table❌ |
We are receiving events from the source files in JSON format, but there are two issues:
Field Extraction:
- Critical fields such as protocol, action, source, destination, etc., are not being identified.
Incomplete Logs:
- Logs appear truncated, starting with partial data (e.g., “urceID:…”) and missing “Reso,” which implies dropped or incomplete events (As far as I understand)
Few logs were received compared to the traffic on Azure Firewall. Attached is a piece of logs showing errors as mentioned in the question.
________________________________________________________________
Environment Details:
• Log Collector: Heavy Forwarder (HF) hosted in Azure. • Data Flow: Logs are being forwarded to Splunk Cloud
Questions:
- Can it be an issue with using storage accounts and not event-hub?
- Could the incomplete logs be due to a configuration issue with the Microsoft Cloud Add-On or possibly related to the data transfer between the storage account and Splunk?
- Has anyone encountered similar issues with field extraction from Azure Firewall JSON logs?
Ultimate Goal:
Receive Azure Firewall Logs with fields extracted as any other firewall logs received by Syslog (Fortinet for example)
Any guidance or troubleshooting suggestions would be much appreciated!
