How to use regex to send events to NullQueue?

[kiran331]

Hi,

How to use regex to send all events related to fw_rule=0 and from a sensor sensor=abcd-f01 to null queue?

sample event:

rec_type=71 app_proto=Unknown client_app=Unknown client_version="" connection_id=0 dest_autonomous_system=0 dest_bytes=0 dest_ip=1.2.3.4 dest_ip_country=0 dest_mask=0 dest_pkts=0 dest_port=80 dest_tos=0 dns_query="" dns_rec_id=0 dns_resp_id=0 dns_ttl=0 event_desc="Flow Statistics" event_sec=1523461342 event_subtype=1 event_type=1003 event_usec=0 file_count=0 first_pkt_sec=1523461341 fw_policy=F01_POLICY fw_rule=0 fw_rule_action=Fastpath fw_rule_reason=N/A has_ipv6=1 http_referrer="" http_response=0 iface_egress=1.2 iface_ingress=P2.3 instance_id=0 ip_layer=0 ip_proto=TCP ips_count=0 last_pkt_sec=0 legacy_ip_address=0.0.0.0 mac_address=0:0:0:0:0:0 monitor_rule_1=N/A monitor_rule_2=N/A monitor_rule_3=N/A monitor_rule_4=N/A monitor_rule_5=N/A monitor_rule_6=N/A monitor_rule_7=N/A monitor_rule_8=0 netbios_domain="" netflow_src=00000000-0000-0000-0000-000000000000 num_ioc=0 rec_type_desc="Connection Statistics" rec_type_simple=RNA referenced_host="" sec_intel_event=No sec_intel_ip=N/A sec_zone_egress=F01_OUTSIDE sec_zone_ingress=F01_INSIDE security_context=00000000000000000000000000000000 sensor=abcd-f01 sinkhole_uuid=00000000-0000-0000-0000-000000000000 snmp_in=0 snmp_out=0 src_autonomous_system=0 src_bytes=0 src_ip=22.33.44.55 src_ip_country=unknown src_mask=0 src_pkts=0 src_port=4382 src_tos=0 ssl_actual_action=Unknown ssl_cert_fingerprint=0000000000000000000000000000000000000000 ssl_cipher_suite=TLS_NULL_WITH_NULL_NULL ssl_expected_action=Unknown ssl_flow_error=0 ssl_flow_flags=0 ssl_flow_messages=0 ssl_flow_status=Unknown ssl_policy_id=00000000000000000000000000000000 ssl_rule_id=0 ssl_server_cert_status="Not Checked" ssl_server_name="" ssl_session_id=0000000000000000000000000000000000000000000000000000000000000000 ssl_ticket_id=0000000000000000000000000000000000000000 ssl_url_category=0 ssl_version=Unknown tcp_flags=0 url="" url_category=Unknown url_reputation="Risk unknown" user="No Authentication Required" user_agent="" vlan_id=0 web_app=Unknown

I tried below in config, it's not working.Is there anything I have to add.

props.conf

[cisco:estreamer:data]
TRANSFORMS-null= setnull

transforms.conf

[setnull]
SOURCE_KEY=_raw
REGEX = (fw_rule=0.*sensor=abcd-f01)
DEST_KEY = queue
FORMAT = nullQueue

[jotne]

[setnull]

Do not use stansa name like this.  What happens if you have two app with samme stansa name, it may give you problem

Use f.eks. 

[remove_firewall_sensor_abcd] 

[haoban]

props.conf and transforms.conf are in heavy forwarder. Still not working, all "cisco:estreamer:data" forwarded to the indexer. I only want "sec_intel_event=Yes" forward to indexer.

/opt/splunk/etc/apps/TA-eStreamer/local# cat props.conf
[cisco:estreamer:data]
TRANSFORMS-send-data-to-null-queue = setnull

/opt/splunk/etc/apps/TA-eStreamer/local# cat transforms.conf
[setnull]
REGEX = (sec_intel_event=Yes)
DEST_KEY = queue
FORMAT = nullQueue

[haoban]

I made a stupid mistake. The NullQueue is like drop something. So the the REGEX should be "sec_intel_event=No'.

Thanks all your help, guys!

[haoban]

I changed the props.conf and transforms.conf as following. But seems no event is forwarded.
===props.conf===
[cisco:estreamer:data]
TRANSFORMS = setnull

===transforms.conf===
[setnull]
REGEX = (sec_intel_event=Yes)
DEST_KEY = queue
FORMAT = nullQueue

[MuS]

Hi there,

the props.conf entry for the TRANSFORMS looks wrong, try something like:

 TRANSFORMS-send-data-to-null-queue = setnull

Also, remember that this must be on the parsing Splunk instance, it needs a restart after the change, and it will only work for new events.

cheers, MuS

[haoban]

I want to forward "Security Intelligence Events", props.conf and transforms.conf are saved in "/opt/splunk/etc/apps/TA-eStreamer/local"

props.conf
[monitor://$SPLUNK_HOME/etc/apps/TA-eStreamer/data]
TRANSFORMS-set = setnull

Transforms.conf
[setnull]
REGEX = (sec_intel_event=Yes)
DEST_KEY = queue
FORMAT = nullQueue

But it doesn't work, who can help me? Thanks!

[ddrillic]

In props.conf something like -

[<sourcetype>]    -- the sourcetype which corresponds to monitor://$SPLUNK_HOME/etc/apps/TA-eStreamer/data
TRANSFORMS = setnull

[haoban]

Do I need to escap the "=" in REGEX such as "REGEX = (sec_intel_event=Yes)"

[MuS]

nope, but you cannot use [monitor://...] in props.conf like @ddrillic wrote, it must be either a host, a source, or a sourcetype in the stanza.

cheers, MuS

[p_gurav · Accepted solution]

Can you try :

[setnull]
 REGEX = fw_rule=0.*sensor=abcd-f01
 DEST_KEY = queue
 FORMAT = nullQueue