FortiGate TA vendor_action not extracting correctly

[ejwade]

I'm running Splunk_TA_fortinet_fortigate version 1.6, and since upgrading a FortiGate to 6.x, my action fields have been incorrectly extracting. Here are the props/transforms:

props.conf
[fgt_traffic]
REPORT-fgt_traffic_vendor_action = action_as_vendor_action

transforms.conf
[action_as_vendor_action]
REGEX = (?:\s|\,)action=\"?([^\s\,\"]+)\"?
FORMAT = vendor_action::$1

Based on the RegEx, vendor_action should be extracted after action=, with optional quotes around the group. The group contains any character EXCEPT a space, comma, or double quote.

Here is a sample log excerpt:

... proto=6 action="deny" ...

Unfortunately, deny is getting extract WITH the quotes:

e.g.
"deny" - (value of the vendor_action field)

I can't seem to figure out why this is happening. Anyone have any suggestions or experience this as well? Note - prior to FortiOS 6.x, the log did not have quotes around the action field value.

[jerryzhao]

splunk should be able to extract the value stripping off the quotes automatically. the one with regex is for extracting vendor actions, which i found had not been even necessary.
let me verify this and let you know

[ejwade]

Unfortunately, vendor_action is used as the upstream field for "action", which is a Network_Traffic CIM data model field.

EVAL-ftnt_action = coalesce(utmaction, vendor_action, vendor_status)

This puts vendor_action into ftnt_action

LOOKUP-fgt_traffic_action = ftnt_action_lookup ftnt_action OUTPUT action

This outputs a CIM normalized action from the lookup file "ftnt_action_lookup" using ftnt_action.

[jerryzhao]

still the original action extracted should be without quotes in the first place. look up only does a translation between action and vendor action.
which splunk version are you using?

[ejwade]

Splunk Enterprise 7.1.4. The action value from the raw log is not an expected CIM field, so I'm assuming that's why the FortiGate TA does the lookup reference.

Unfortunately, "deny" (in quotes) is not present in the lookup table, so it can't output the action.

One workaround would be to duplicate the values in the lookup table with quotes surrounding ftnt_action field values. However, I'd like to figure out why the quotes are there in the first place.

[jerryzhao]

yes. vendor_action is for CIM model, i wrote that. but anyway that is not what we should worry about.
as i replied to your email, i have no issue with 6.0 FOS logs with quotes. Do you have other TA for fortigate installed than our TA? i remember there is one that comes with enterprise security installation, you have to uninstall it.
have your traffic logs, for example, been transformed to fgt_traffic sourcetype? if not, that is an indication that our TA is not taking effect.

[ejwade]

Yes - logs are being transformed to their respective sourcetypes (fgt_traffic for traffic). There aren't any other FortiGate TAs installed on the search head.

I should mention we are logging two FortiGates, one with FOS 5.4 and the other with 6.0. The 5.4 log does not have quotes, while the 6.0 does. The quotes do not show up for 5.4 action field values, but they do in the 6.0. Both are using the same configuration (same TAs).

Based on the transforms.conf, it looks like fields are extracted per this stanza:
[field_extract]
DELIMS = "\ ,", "="

After this stanza is [action_as_vendor_action], which I assume overrides the one above for the raw "action" field. When I tried disabling [action_as_vendor_action], the quotes went away, but these aren't the values I'm after (the ones in the lookup table).

I hope that makes sense.

[jerryzhao]

our current TA works for both 5.4, 5.6 and 6.0.
as i included in my email reply, the screen capture shows the action="deny" was correctly converted to blocked for action and vendor_action.
i verified this on a new installation with the app/TA on splunk base. please make sure the TA are installed on all forwarder, indexer and searchheads, unmodified. My splunk server is 7.2.4 though but double quotes have been verified long ago since 5.6 introduced double quotes in logs.
can you try reinstalling them?