How to fix firewall data Parsing issue from the syslogs?

[Hemnaath]

Hi All,

We are facing an data parsing issue with the check point firewall logs.

Problem Details :

index=firewall sourcetype="opsec*" | table service dest_port

When doing the above search, we noticed that for some of the events the dest_port fields are left empty, when investigated the issue we found that there are many values in the service field that are resolved services like "domain-udp" instead of port numbers like "9099, Which means that they won't be parsed into dest_port.

Only the values in "service" with an actual number are parsed as "dest_port" while values in "service" that are resolved as words are not included. 

Event Details1:

time=1522934769|loc=1701735|fileid=1522933895|action=accept|orig=test02fw.xxxx.com|i/f_dir=inbound|i/f_name=bond0.905|has_accounting=0|logId=-1|log_type=log|log_sequence_num=0|is_first_for_luuid=131072|log_version=1|uuid=<5ac623f1,00000001,14fcd00a,c0000000>|product=VPN-1 & FireWall-1|inzone=Internal|outzone=External|rule=35|rule_uid={07FA4A1E-BF88-4F51-9428-3BAB6AABE07A}|rule_name=External DNS|service_id=domain-udp|src=sgosixxxx.xxxx.com|s_port=26108|dst=95.X,X,X|service=domain-udp|proto=udp|xlatesrc=sgoxx-cluster_External_Interface|xlatesport=25175|xlatedport=Unknown|NAT_rulenum=13|NAT_addtnl_rulenum=1|__policy_id_tag=product=VPN-1 & FireWall-1[db_tag={E0D0044A-9CB4-8248-86D9-9DE7A3F332DF};mgmt=smart03;date=1518285268;policy_name=Efficiency_Hubs]|origin_sic_name=CN=test02fw.xxxx.com,O=dron01.xxxx.com.eovv25

Event Action

Field Value
service =domain-udp

Event Details2:

time=1522935417|loc=4222603|fileid=1522933895|action=drop|orig=test02fw.xxxxx.com|i/f_dir=inbound|i/f_name=bond1.900|has_accounting=0|logId=-1|log_type=log|log_sequence_num=0|is_first_for_luuid=131072|log_version=1|uuid=<00000000,00000000,00000000,00000000>|product=VPN-1 & FireWall-1|inzone=External|outzone=Local|rule=7|rule_uid={B68CC4E7-E448-40B8-8F94-379D9AA8C62B}|rule_name=FW Stealth Rule|src=X.X.X.X|s_port=35269|dst=sgoingfw-cluster_External_Interface|service=33438|proto=udp|__policy_id_tag=product=VPN-1 & FireWall-1[db_tag={E0D0044A-9CB4-8248-86D9-9DE7A3F332DF};mgmt=smart03;date=1518285268;policy_name=Efficiency_Hubs]|origin_sic_name=CN=test02fw.xxxxx.com,O=daron01.xxxxx.com.eovv25

Event Action

Field Value
service =33438
dest_port=33438

Questions

1)Instead of the word of the service, How to insert value of the port number in the logs? Is there way in splunk configuration?

Props.conf

EVAL-dest_port = if(isnum(service),service,null)

Kindly guide me on this.

[robgora_deloitt]

Have you tried using the Splunk Add-on for Check Point OPSEC LEA? I know using add-ons like these will normally help with issues like this.

https://splunkbase.splunk.com/app/3197/

[Hemnaath]

Hi Robgora, thanks for your effort, we are using the same configuration as given in this add-on, but has I had commented earlier we could notice that for some of the events the dest_port fields are left empty and this due to the values in "service" field with an actual number are parsed as "dest_port" while values in "service" that are resolved as words are not included.

So kindly guide me is there a way in splunk configuration which can be applied to overcome this issue.

thanks in advance.

[robgora_deloitt]

It looks like what you are seeing is by design. After looking at your sample log entries, the fist one looks to be going from internal to external (|inzone=Internal|outzone=External) and your second is external to internal (inzone=External|outzone=Local). This could be why you are seeing a service port as the destination for the second example and not the first. Based on what is in your logs, and what is in the props.conf of the Splunk App, it is showing correctly.

If you are using the Add-on, then you would have to create a new props.conf and add in the modification you would want to overwrite what is in the props.conf in the default folder.

[Hemnaath]

Hi robgora_deloitte, thanks for your effort, could you please guide me on how to fix this issue by applying the customized syntax in props.conf file.

[robgora_deloitt]

Do you know what field you want to replace it with? It doesn't look like there are any other fields in the data that I can see.

I would probably contact Splunk Support since this App was built by them or talk to your vendor about the data coming out.

Once you know the field that you want to use then you can create a props.conf under the local folder that has the same Eval statement from about but set it to another field other than service.

[Hemnaath]

Hey we want to replace the value not the field, currently the values in "service" with an actual number are parsed as "dest_port" while values in "service" that are resolved as words are not included. Instead of the word of the service, we want insert value of the port number in the logs. So is that possible by changing the syntax in the props.conf.

Syntax available in Props.conf for opsec

FIELDALIAS-dest_for_opsec = dst as dest, dst as dest_ip
FIELDALIAS-src_for_opsec = src as src_ip
FIELDALIAS-direction_for_opsec = i_f_dir as direction
FIELDALIAS-dvc_for_opsec       = orig as dvc
FIELDALIAS-event_id_for_opsec  = loc as event_id 
FIELDALIAS-vendor_product = product as vendor_product
FIELDALIAS-vendor_action = action as vendor_action
FIELDALIAS-dest_interface = outbound_interface as dest_interface
FIELDALIAS-src_interface = inbound_interface as src_interface
FIELDALIAS-icmp_code = ICMP as icmp_code
FIELDALIAS-icmp_type = ICMP_Type as icmp_type
FIELDALIAS-packets_in = client_inbound_packets as packets_in
FIELDALIAS-packets_out = client_outbound_packets as packets_out
FIELDALIAS-src_zone  = inzone as src_zone
FIELDALIAS-dest_zone  = outzone as dest_zone

EVAL-dvc_ip = if(match(orig,"^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$"), orig, null)
EVAL-rule = case(isnull(rule),policy_name,(not isnum(rule)),rule,isnotnull(policy_name),policy_name,1==1,null)
EVAL-rule_id =  if(isnum(rule),rule,null)
EVAL-src_port = if(isnum(s_port),s_port,null)
EVAL-src_svc = if(isnum(s_port),null,s_port)
EVAL-dest_port = if(isnum(service),service,null)
EVAL-dest_svc = if(isnum(service),null,service)
EVAL-transport = case(lower(proto)=="tcp","tcp",lower(proto)=="udp","udp",lower(proto)=="icmp","icmp",1==1,null)
EVAL-received_bytes = if (received_bytes < 0, 2147483647, received_bytes)
EVAL-sent_bytes = if(sent_bytes < 0, 2147483647, sent_bytes)
EVAL-bytes_in = if (received_bytes < 0, 2147483647, received_bytes)
EVAL-bytes_out = if(sent_bytes < 0, 2147483647, sent_bytes)
EVAL-bytes = if(bytes < 0, 2147483647, bytes)
EVAL-client_inbound_bytes = if(client_inbound_bytes < 0, 2147483647, client_inbound_bytes)
EVAL-client_outbound_bytes = if(client_outbound_bytes < 0, 2147483647,client_outbound_bytes)
EVAL-server_inbound_bytes = if(server_inbound_bytes < 0, 2147483647, server_inbound_bytes)
EVAL-server_outbound_bytes = if(server_outbound_bytes < 0, 2147483647,server_outbound_bytes)

Note: Only partial syntax is pasted in the comment.