Splunk Enterprise

Unable to drop/filter certain events on Heavy Forwarder using props.conf and transform.conf

markpdeakin
Explorer

Hi Splunk Community,

I am seeking assistance on what should be a relatively simple task - to drop/filter particular events on a heavy forwarder node using props.conf/transforms.conf. I have successfully implemented filtering/dropping events within the environment many times before so I am surprised how how difficult to make work this particular transform/prop is. It is not working as expected despite the string matching regex working successfully when testing on the Splunk GUI, the Splunk pcregextest CLI tool and regex101.com.

The log events are from a Cisco Firepower firewall. They are sent via syslog from the firewall to a Linux syslog server where they are written to disk and then picked up by the SplunkUniversalForwarder, sent to a Heavy Forwarder node before being sent to an Index node.

I have tried many permutations of the props.conf/transforms.conf below including sending events which do NOT contain the specified words to the nullQueue or sending everything to the nullQueue then filtering events which DO match the regex to the indexQueue, various changes to the regex etc. the result is that the Splunk index node (and the resultant index) will either receive every event or no events at all (depending on the test).

Any help or tips to assist debugging this problem will be greatly appreciated.

Thanks,

 

What I am trying to achieve with props.conf/transforms.conf?

- Drop any log events which do not contain the word "URLSICategory" or "DNSSICategory" or "IPReputationSICategory"

 

inputs.conf (Linux syslog host - universal forwarder)

[monitor:///var/log/firewall/firewall_test.log]
disabled=false
index=ngfw_security_intelligence
sourcetype=security_syslog

 

transforms.conf (heavy forwarder)

[allsetnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[ngfw_si_events_whitelist]
REGEX = ((?:URLSICategory|DNSSICategory|IPReputationSICategory))
DEST_KEY = queue
FORMAT = indexQueue

 

props.conf (heavy forwarder)

[ngfw_security_intelligence]
TRANSFORMS-ngfw-drop = allsetnull, ngfw_si_events_whitelist

 

Other regex tried:

- Match lines which do NOT contain the words (then nullQueue them)

(?s)^((?!URLSICategory|DNSSICategory|IPReputationSICategory).)*$

- Match lines which DO contain the words (then indexQueue them)

((?:URLSICategory|DNSSICategory|IPReputationSICategory))
(?:URLSICategory|DNSSICategory|IPReputationSICategory)
(URLSICategory|DNSSICategory|IPReputationSICategory)
URLSICategory|DNSSICategory|IPReputationSICategory

 

Testing done (GUI):

index=ngfw_security_intelligence | regex _raw="^((?!URLSICategory|DNSSICategory|IPReputationSICategory).)*$"
index=ngfw_security_intelligence | regex _raw="(?s)^((?!URLSICategory:|DNSSICategory:|IPReputationSICategory:).)*$"

 

Testing done (CLI):

./splunk cmd pcregextest mregex="^((?!URLSICategory|DNSSICategory|IPReputationSICategory).)*$" test_str="Destination, IPReputationSICategory: Global-Blacklist_1"

 

Testing done (regex101.com):

https://regex101.com/r/MDQqBx/1

https://regex101.com/r/rLDxHr/1

 

The log data (anonymized for this post):

Rcvd:2021-04-15T11:17:46.673993+10:00 From:firewall-1.site.com Time:2021-04-15T01:17:45 Host:firewall-1 Pri:alert.info Msg: %FTD-6-430002: EventPriority: High, DeviceUUID: 00000000-0000-0000-0000-000000000001, InstanceID: 16, FirstPacketSecond: 2021-04-15T01:17:45Z, ConnectionID: 35916, AccessControlRuleAction: Block, AccessControlRuleReason: IP Block, SrcIP: 10.0.0.8, DstIP: 1.2.3.9, SrcPort: 16403, DstPort: 16386, Protocol: udp, IngressInterface: Inside, EgressInterface: Outside, IngressZone: INSIDE, EgressZone: OUTSIDE, IngressVRF: Global, EgressVRF: Global, ACPolicy: ACP-XX-20210329, Prefilter Policy: PFP-XX-20210329, InitiatorPackets: 1, ResponderPackets: 0, InitiatorBytes: 58, ResponderBytes: 0, NAPPolicy: No Rules Active, SecIntMatchingIP: Destination, IPReputationSICategory: Global-Blacklist_1
Rcvd:2021-04-15T11:17:49.924536+10:00 From:firewall-1.site.com Time:2021-04-15T01:17:49 Host:firewall-1 Pri:alert.info Msg: %FTD-6-430003: EventPriority: Low, DeviceUUID: 00000000-0000-0000-0000-000000000001, FirstPacketSecond: 2021-04-15T01:17:49Z, ConnectionID: 0, AccessControlRuleAction: Allow, SrcIP: 10.0.0.1, DstIP: 1.2.3.4, SrcPort: 54102, DstPort: 443, Protocol: tcp, IngressInterface: Inside, EgressInterface: Outside, IngressZone: INSIDE, EgressZone: OUTSIDE, IngressVRF: Global, EgressVRF: Global, ACPolicy: ACP-XX-20210329, AccessControlRuleName: From-IPv4-allowed-users, Prefilter Policy: PFP-XX-20210329, User: Not Found, ConnectionDuration: 0, InitiatorPackets: 12, ResponderPackets: 15, InitiatorBytes: 1419, ResponderBytes: 12575, NAPPolicy: No Rules Active
Rcvd:2021-04-15T12:53:40.111154+10:00 From:firewall-1.site.com Time:2021-04-15T02:53:39 Host:firewall-1 Pri:alert.info Msg: %FTD-6-430003: EventPriority: Low, DeviceUUID: 00000000-0000-0000-0000-000000000001, FirstPacketSecond: 2021-04-15T02:53:39Z, ConnectionID: 0, AccessControlRuleAction: Allow, SrcIP: 10.0.0.2, DstIP: 1.2.3.5, SrcPort: 48012, DstPort: 443, Protocol: tcp, IngressInterface: Inside, EgressInterface: Outside, IngressZone: INSIDE, EgressZone: OUTSIDE, IngressVRF: Global, EgressVRF: Global, ACPolicy: ACP-XX-20210329, AccessControlRuleName: From-IPv4-allowed-users, Prefilter Policy: PFP-XX-20210329, User: Not Found, ConnectionDuration: 0, InitiatorPackets: 10, ResponderPackets: 15, InitiatorBytes: 1678, ResponderBytes: 12575, NAPPolicy: No Rules Active
Rcvd:2021-04-15T12:53:40.112896+10:00 From:firewall-1.site.com Time:2021-04-15T02:53:40 Host:firewall-1 Pri:alert.info Msg: %FTD-6-430003: EventPriority: Low, DeviceUUID: 00000000-0000-0000-0000-000000000001, FirstPacketSecond: 2021-04-15T02:53:40Z, ConnectionID: 0, AccessControlRuleAction: Allow, SrcIP: 10.0.0.3, DstIP: 1.2.3.6, SrcPort: 48019, DstPort: 443, Protocol: tcp, IngressInterface: Inside, EgressInterface: Outside, IngressZone: INSIDE, EgressZone: OUTSIDE, IngressVRF: Global, EgressVRF: Global, ACPolicy: ACP-XX-20210329, AccessControlRuleName: From-IPv4-allowed-users, Prefilter Policy: PFP-XX-20210329, User: Not Found, ConnectionDuration: 0, InitiatorPackets: 11, ResponderPackets: 15, InitiatorBytes: 1678, ResponderBytes: 12575, NAPPolicy: No Rules Active
Rcvd:2021-04-15T12:53:40.123993+10:00 From:firewall-1.site.com Time:2021-04-15T01:17:45 Host:firewall-1 Pri:alert.info Msg: %FTD-6-430002: EventPriority: High, DeviceUUID: 00000000-0000-0000-0000-000000000002, InstanceID: 16, FirstPacketSecond: 2021-04-15T01:17:45Z, ConnectionID: 35916, AccessControlRuleAction: Block, AccessControlRuleReason: IP Block, SrcIP: 10.0.0.4, DstIP: 1.2.3.7, SrcPort: 16403, DstPort: 16386, Protocol: udp, IngressInterface: Inside, EgressInterface: Outside, IngressZone: INSIDE, EgressZone: OUTSIDE, IngressVRF: Global, EgressVRF: Global, ACPolicy: ACP-XX-20210329, Prefilter Policy: PFP-XX-20210329, InitiatorPackets: 1, ResponderPackets: 0, InitiatorBytes: 58, ResponderBytes: 0, NAPPolicy: No Rules Active, SecIntMatchingIP: Destination, IPReputationSICategory: Global-Blacklist_1
Rcvd:2021-04-15T12:53:40.130404+10:00 From:firewall-2.site.com Time:2021-04-15T12:53:40 Host:firewall-2 Pri:local4.err Msg::Apr 15 02:53:40 UTC: %FTD-session-3-106014: Deny inbound icmp src Inside:10.1.0.1 dst nlp_int_tap:169.254.1.2 (type 3, code 3)

0 Karma
1 Solution

markpdeakin
Explorer

I thought I would comment and provide the solution for this in case other community users run into a similar problem. I was able to get this working by specifying source::<source> in props.conf rather than the index name. Index names are not supported in props.conf, the existing entries we had which I thought were operating on index name were actually operating on the sourcetype which had been set on the Universal Forwarder client in inputs.conf with a name similar to the index name.

 

https://docs.splunk.com/Documentation/Splunk/Latest/Admin/Propsconf

<spec> can be:
1. <sourcetype>, the source type of an event.
2. host::<host>, where <host> is the host, or host-matching pattern, for an
                 event.
3. source::<source>, where <source> is the source, or source-matching
                     pattern, for an event.
4. rule::<rulename>, where <rulename> is a unique name of a source type
                     classification rule.
5. delayedrule::<rulename>, where <rulename> is a unique name of a delayed
                            source type classification rule.
                            These are only considered as a last resort
                            before generating a new source type based on the
                            source seen.

 

The correct entry for props.conf is:

[source::/var/log/firewall/firewall_test.log]
TRANSFORMS-ngfw-drop-non-si-events = allsetnull, ngfw_si_events_whitelist

 

The correct entries for transforms.conf are:

[allsetnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[ngfw_si_events_whitelist]
REGEX = (URLSICategory|DNSSICategory|IPReputationSICategory)
DEST_KEY = queue
FORMAT = indexQueue

View solution in original post

Tags (1)
0 Karma

markpdeakin
Explorer

I thought I would comment and provide the solution for this in case other community users run into a similar problem. I was able to get this working by specifying source::<source> in props.conf rather than the index name. Index names are not supported in props.conf, the existing entries we had which I thought were operating on index name were actually operating on the sourcetype which had been set on the Universal Forwarder client in inputs.conf with a name similar to the index name.

 

https://docs.splunk.com/Documentation/Splunk/Latest/Admin/Propsconf

<spec> can be:
1. <sourcetype>, the source type of an event.
2. host::<host>, where <host> is the host, or host-matching pattern, for an
                 event.
3. source::<source>, where <source> is the source, or source-matching
                     pattern, for an event.
4. rule::<rulename>, where <rulename> is a unique name of a source type
                     classification rule.
5. delayedrule::<rulename>, where <rulename> is a unique name of a delayed
                            source type classification rule.
                            These are only considered as a last resort
                            before generating a new source type based on the
                            source seen.

 

The correct entry for props.conf is:

[source::/var/log/firewall/firewall_test.log]
TRANSFORMS-ngfw-drop-non-si-events = allsetnull, ngfw_si_events_whitelist

 

The correct entries for transforms.conf are:

[allsetnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[ngfw_si_events_whitelist]
REGEX = (URLSICategory|DNSSICategory|IPReputationSICategory)
DEST_KEY = queue
FORMAT = indexQueue

Tags (1)
0 Karma

aasabatini
Motivator

Hi @markpdeakin 

 

maybe is a stupid question but why do you have two regex on your transforms stanza?

[ngfw_si_events_whitelist]
REGEX = ((?:URLSICategory|DNSSICategory|IPReputationSICategory))
REGEX = IPReputationSICategory
DEST_KEY = queue
FORMAT = indexQueue

Couldn't he work with just one? maybe the first

“The answer is out there, Neo, and it’s looking for you, and it will find you if you want it to.”

markpdeakin
Explorer

Hi aasabatini,

Thanks for picking up on this error, I have update the original post to only include a single regex line. The second regex line was actually #commented out on my Splunk heavy forwarder system and I messed up the formatting when copying the configs across to this post.

Thanks,

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...