Getting Data In

Cisco eStreamer eNcore - filtering not working

halfreeman
New Member

Good day,

I'm trying to filter connection events from FMC eStreamer, i.e. I do not need Allowed connections in Splunk. I thought I had lurked enough into Splunk documentation and Community, but I guess - not. 😞 So I need help from senior admins.

I had created in /opt/splunk/etc/apps/TA-eStreamer/local
props.conf
[cisco:estreamer:data]
rename = fmc
TRANSFORMS-send-data-to-null-queue = setnull

transforms.conf
[setnull]
REGEX = (fw_rule_action=Allow)
DEST_KEY = queue
FORMAT = nullQueue

Restarted Splunk, and yet I still get
rec_type=71 file_count=0 client_app="SSL client" mac_address=00:00:00:00:00:00 dest_ip=... dest_port=443 sec_intel_ip=N/A
...
ssl_expected_action=Unknown app_proto=HTTPS ssl_server_name="" ssl_cert_fingerprint=0000000000000000000000000000000000000000 has_ipv6=1 fw_rule_action=Allow ssl_rule_id=0
...

I'm using latest eStreamer eNcore
https://splunkbase.splunk.com/app/3662/

0 Karma
1 Solution

dkeck
Influencer

HI,
you can discard whole rec_type like this:

transforms

[discard_rec_type_71]
REGEX=^rec_type=71
DEST_KEY=queue
FORMAT=nullQueue

Props.conf

[cisco:estreamer:data]
TRANSFORMS-rec_type71=discard_rec_type_71

Or do you try to discard fields within the rec_type=71 events? This is possible as well.

View solution in original post

dkeck
Influencer

HI,
you can discard whole rec_type like this:

transforms

[discard_rec_type_71]
REGEX=^rec_type=71
DEST_KEY=queue
FORMAT=nullQueue

Props.conf

[cisco:estreamer:data]
TRANSFORMS-rec_type71=discard_rec_type_71

Or do you try to discard fields within the rec_type=71 events? This is possible as well.

halfreeman
New Member

Alas rec_type=71 are for all kinds of messages - Allow, Trust, Block. Block with reset.

I.e.
rec_type=71 file_count=0 client_app="SSL client" mac_address=00:00:00:00:00:00 dest_ip=...
...
ssl_server_name="" ssl_cert_fingerprint=0000000000000000000000000000000000000000 has_ipv6=1 fw_rule_action="Block with reset" ssl_rule_id=0 netbios_domain="" fw_rule="Bacis access - Deny Traffic"
...

Actually I either need to discard messages with fw_rule_action= equals Allow or Trust or keep only
Block and *Block with reset"

0 Karma

dkeck
Influencer

You could try :

[discard_rec_type_71]
REGEX=fw_rule_action=("Allow"|"Trust")
DEST_KEY=queue
FORMAT=nullQueue

0 Karma

halfreeman
New Member

Actually - this one worked just need to remove quote marks REGEX=fw_rule_action=(Allow|Trust).

0 Karma

dkeck
Influencer

Great pls accept the answer 🙂

0 Karma

dkeck
Influencer

Small update, if you are interessed:

you can add this to make sure its only valid for rec_type 71

^rec_type=71.*?fw_rule_action=(Allow|Trust)

0 Karma

halfreeman
New Member

No success.

Btw, just in case I tried "REGEX=^rec_type=71" and it worked - all such messages were filter out. So in general the idea is correct.

The question is in specific REGEX?

Here is whole typical "Allow" message:
rec_type=71 web_app=Weborama dest_port=443 file_count=0 dest_autonomous_system=0 sec_intel_ip=N/A netflow_src=00000000-0000-0000-0000-000000000000 http_response=0 dest_bytes=992 user_agent="" dns_resp_id=0 event_type=1003 dns_rec_id=0 client_app="SSL client" ssl_flow_error=0 legacy_ip_address=0.0.0.0 event_usec=0 vlan_id=0 ssl_actual_action=Unknown dns_query="" dest_ip_country=france snmp_out=0 ssl_flow_status=Unknown event_desc="Flow Statistics" src_mask=0 num_ioc=0 rec_type_desc="Connection Statistics" sensor=Onego event_subtype=1 security_context=00000000000000000000000000000000 sinkhole_uuid=00000000-0000-0000-0000-000000000000 fw_rule_action=Allow dest_pkts=4 fw_policy=00000000-0000-0000-0000-00005c47a1f6 url=https://adv.solution.weborama.fr tcp_flags=0 mac_address=00:00:00:00:00:00 has_ipv6=1 ssl_server_cert_status="Not Checked" fw_rule_reason=N/A ssl_version=Unknown dns_ttl=0 ssl_rule_id=0 app_proto=HTTPS ssl_flow_messages=0 ssl_cert_fingerprint=0000000000000000000000000000000000000000 iface_ingress=inside src_tos=0 referenced_host="" iface_egress=outside monitor_rule_1=N/A monitor_rule_3=N/A monitor_rule_2=N/A monitor_rule_5=N/A monitor_rule_4=N/A monitor_rule_7=N/A monitor_rule_6=N/A monitor_rule_8=0 instance_id=1 dest_ip=91.216.195.18 ip_layer=0 last_pkt_sec=1548241758 ssl_session_id=0000000000000000000000000000000000000000000000000000000000000000 src_pkts=6 sec_intel_event=No dest_mask=0 snmp_in=0 rec_type_simple=RNA first_pkt_sec=1548241758 fw_rule="Extended access" ssl_policy_id=00000000000000000000000000000000 url_category=Uncategorized url_reputation="Risk unknown" ssl_url_category=0 ip_proto=TCP ssl_flow_flags=0 src_bytes=2022 event_sec=1548241938 sec_zone_ingress=Sanctum ssl_cipher_suite=TLS_NULL_WITH_NULL_NULL netbios_domain="" src_autonomous_system=0 client_version="" src_ip_country=unknown ssl_ticket_id=0000000000000000000000000000000000000000 dest_tos=0 src_port=52189 http_referrer="" sec_zone_egress=Exodus src_ip=192.168.111.111 user=blahblahuser ips_count=0 ssl_server_name="" ssl_expected_action=Unknown connection_id=28348

0 Karma

rajashaey
Engager

hello, am looking for reference of field "rec_type" and what is actually means?
I tried searching cisco documentation but no luck. Please share with me? the link @halfreeman @dkeck 

0 Karma

dkeck
Influencer

https://www.cisco.com/c/en/us/td/docs/security/firesight/540/api/estreamer/EventStreamerIntegrationG...

maybe this helps?

I am not really familiar with estreamer in details. What I understood from a splunk perspective is that, rec_type is the main identifier for the firewall events. The TA also use it to break the events. In the python script you are able to filter out rec_types and fields, based on rec_types.

David

0 Karma

dkeck
Influencer

Did you try this?

Please accept the answer if it helped

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...