topic Re: Fields extraction problem in Splunk Search

Fields extraction problem

himang2c — Mon, 28 Sep 2020 12:51:25 GMT

HI Experts.

I did fields extraction in regexr, The fields matching in regexr is no problem.
But, On splunk , we can see only EXTRACT-Security_Firewall-threat_00. This logs is paloalto logs.
EXTRACT-Security_Firewall-threat_01 cannot see on splunk fields. what is problem ?

EXTRACT-Security_Firewall-threat_00 = ^\w+\s+\d+\s+\d+:\d+:\d+\s+\d+.\d+.\d+.\d+\s+\w+\s+\d+\s+\d+:\d+:\d+\s+(?<future_use1>[^,]+),+(?<receive_time>[^,]+),+(?<serial_number>[^,]+),+(?<type>[^,]+),+(?<log_subtype>[^,]+),+(?<future_use2>[^,]+),+(?<generated_time>[^,]+),+(?<src_ip>[^,]+),+(?<dst_ip>[^,]+),+(?<nat_src_ip>[^,]+),+(?<nat_dst_ip>[^,]+),+(?<rule_name>[^,]+),(?<src_user>\w*)\,(?<dst_user>\w*)\,+(?<application>[^,]+),+(?<virtual_system>[^,]+),+(?<src_zone>[^,]+),+(?<dst_zone>[^,]+),+(?<ingress_interface>[^,]+),+(?<egress_interface>[^,]+),+(?<log_forwarding_profile>[^,]+),+(?<future_use3>[^,]+),+(?<session_id>[^,]+),+(?<repeat_count>[^,]+),+(?<src_port>[^,]+),+(?<dst_port>[^,]+),+(?<nat_src_port>[^,]+),+(?<nat_dst_port>[^,]+),+(?<flags>[^,]+),+(?<protocol>[^,]+),+(?<action>[^,]+),"+(?<misc>[^,]+)\"

EXTRACT-Security_Firewall-threat_01 = \",(?<threat_id>[^,]+),+(?<category>[^,]+),+(?<severity>[^,]+),+(?<direction>[^,]+),+(?<sequence_number>[^,]+),+(?<action_flags>[^,]+),+(?<src_location>[^,]+),+(?<dst_location>[^,]+),+(?<future_use4>[^,]+),+(?<content_type>[^ ].*)

Re: Fields extraction problem

Drainy — Fri, 23 Nov 2012 09:05:21 GMT

That looks like a dreadful regex, also we need some example data to help

Re: Fields extraction problem

jonuwz — Fri, 23 Nov 2012 11:23:26 GMT

Try replacing -threat with _threat.

I've read (but not tested) that using - in your extract name causes problems because '-' is a delimiter.

In that case both your extracts woould be called Security_Firewall

Re: Fields extraction problem

stefandagerman — Fri, 23 Nov 2012 21:48:39 GMT

I believe that if both EXTRACTs apply to the same source, sourcetype or host, you will get the outcome you see. The documentation for the props.conf spec (http://docs.splunk.com/Documentation/Splunk/5.0/Admin/Propsconf#props.conf.spec) states that you need to use REPORT instead of EXTRACT if - amongst other reasons - you want to:

* Apply more than one field-extracting regular expression to the same source, source
type, or host. This can be necessary in cases where the field or fields that you want
to extract from a particular source, source type, or host appear in two or more very
different event patterns.

Can you try REPORT and (pun not intended) report back, please?

Re: Fields extraction problem

Drainy — Fri, 23 Nov 2012 21:53:13 GMT

Not quite. Basically the difference is that EXTRACT allows you to put an inline regular expression into your props stanzas. REPORT just references a stanza inside transforms. In this case you could do EXTRACT-rule1 = ([^\d+]), EXTRACT-rule2 = ([^\d+]) on different lines. With REPORT you would just put REPORT-rules = rule1,rule2 and inside transforms you would define a rule1 and rule2 stanza. Either approach would work