Getting Data In

Unable to use nullqueue properly for a Storage Account input

att35
Builder

Hi everyone,

We are pulling Firewall data from a Storage Account containing several categories. There is one specific category, AZFWDnsQuery which need to be dropped. 

I tested the regex in the search as well as on regex101. It is successfully matching only those specific events with this category.

But once deployed, Splunk starts dropping all events from this input, including for other categories that do not match the regex. 

Sample events

{ "time": "2025-02-27T18:46:08.307710+00:00", "resourceId": "/SUBSCRIPTIONS/xxxxxx/xxx/Path", "properties": {"SourceIp":"x.x.x.x","SourcePort":25208,"QueryId":51787,"QueryType":"A","QueryClass":"IN","QueryName":"google.com","Protocol":"udp","RequestSize":48,"DnssecOkBit":false,"EDNS0BufferSize":512,"ResponseCode":"NOERROR","ResponseFlags":"qr,rd,ra","ResponseSize":94,"RequestDurationSecs":0.007257565,"ErrorNumber":0,"ErrorMessage":""}, "category": "AZFWDnsQuery"}
{ "time": "2025-02-27T18:46:08.307329+00:00", "resourceId": "/SUBSCRIPTIONS/xxxxxx/xxx/Path", "properties": {"SourceIp":"x.x.x.x","SourcePort":62730,"QueryId":16828,"QueryType":"A","QueryClass":"IN","QueryName":"google.com","Protocol":"udp","RequestSize":35,"DnssecOkBit":false,"EDNS0BufferSize":512,"ResponseCode":"NOERROR","ResponseFlags":"qr,rd,ra","ResponseSize":68,"RequestDurationSecs":0.012227477,"ErrorNumber":0,"ErrorMessage":""}, "category": "AZFWDnsQuery"}
{ "time": "2025-02-27T18:46:08.307262+00:00", "resourceId": "/SUBSCRIPTIONS/xxxxxx/xxx/Path", "properties": {"SourceIp":"x.x.x.x","SourcePort":45452,"QueryId":25241,"QueryType":"A","QueryClass":"IN","QueryName":"google.com","Protocol":"udp","RequestSize":35,"DnssecOkBit":false,"EDNS0BufferSize":512,"ResponseCode":"NOERROR","ResponseFlags":"qr,rd,ra","ResponseSize":68,"RequestDurationSecs":0.008439891,"ErrorNumber":0,"ErrorMessage":""}, "category": "AZFWDnsQuery"}
{ "time": "2025-02-27T18:46:08.307129+00:00", "resourceId": "/SUBSCRIPTIONS/xxxxxx/xxx/Path", "properties": {"SourceIp":"x.x.x.x","SourcePort":14846,"QueryId":3916,"QueryType":"A","QueryClass":"IN","QueryName":"google.com","Protocol":"udp","RequestSize":35,"DnssecOkBit":false,"EDNS0BufferSize":512,"ResponseCode":"NOERROR","ResponseFlags":"qr,rd,ra","ResponseSize":68,"RequestDurationSecs":0.009026804,"ErrorNumber":0,"ErrorMessage":""}, "category": "AZFWDnsQuery"}

 

Regex

\"category\":\s\"AZFWDnsQuery\"

 

Here is how props.conf and transforms.conf are configured.

 

[sourcetype]
TRANSFORMS-null=DropFirewallEvents
	
[DropFirewallEvents]
REGEX=_raw=\"category\":\s\"AZFWDnsQuery\"
DEST_KEY=queue
FORMAT=nullQueue

 

What could be doing wrong here for Splunk to drop every event from this input?

Thanks

Labels (1)
Tags (1)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

The REGEX attribute must contain a capturing group, even if it's not used.  Also, no need to reference _raw.

[DropFirewallEvents]
REGEX= ("category":\s"AZFWDnsQuery")
DEST_KEY=queue
FORMAT=nullQueue

 

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

The REGEX attribute must contain a capturing group, even if it's not used.  Also, no need to reference _raw.

[DropFirewallEvents]
REGEX= ("category":\s"AZFWDnsQuery")
DEST_KEY=queue
FORMAT=nullQueue

 

---
If this reply helps you, Karma would be appreciated.

att35
Builder

Thanks @richgalloway 

This worked.

Get Updates on the Splunk Community!

Splunk Decoded: Service Maps vs Service Analyzer Tree View vs Flow Maps

It’s Monday morning, and your phone is buzzing with alert escalations – your customer-facing portal is running ...

What’s New in Splunk Observability – September 2025

What's NewWe are excited to announce the latest enhancements to Splunk Observability, designed to help ITOps ...

Fun with Regular Expression - multiples of nine

Fun with Regular Expression - multiples of nineThis challenge was first posted on Slack #regex channel ...