Splunk Add-on for Microsoft Windows: Why am I getting error "Missing FORMAT for: transform_name='Security_ID_as_src_nt_domain'"?

[marksnelling]

Hi,
I'm running the latest Splunk indexer and forwarders (6.2.1) with my indexer on Linux and my forwarders on Windows 2008 R2. I've also deployed the Splunk Add-on for Microsoft Windows (4.7.3) on both the indexer and forwarders.
I'm seeing a lot of the following errors in my splunkd.log

WARN SearchOperator:kv - Invalid key-value parser, ignoring it, transform_name='Security_ID_as_src_nt_domain'
WARN SearchOperator:kv - Missing FORMAT for: transform_name='Security_ID_as_src_nt_domain'

What are these errors from and how can I fix them?

[dflodstrom]

I found this post after discovering this issue on our deployment. I upgraded to the latest version of Splunk_TA_windows, 4.7.5, and the bug still exists. As a workaround I've added a .../local/transforms.conf with the affected stanzas modified as follows:

[Security_ID_as_src_nt_domain]
REGEX = (.+)(\\)
FORMAT = src_nt_domain::"$1"

Of course you can use a non-capturing group to achieve the same result. Now when I check this field transformation in the GUI the regex and format are correct.

[Masa]

This issue is happening in Splunk_TA_windows transforms.conf.

At least, I can tell v4.7.3 already had this issue. Now v4.7.5 we still have this issue.
So, I'm posting a workaround here.

Here is an example of the problematic regex in the default transforms.conf in the add-on

[Security_ID_as_dest_nt_domain]
SOURCE_KEY = Security_ID
REGEX = (.+)\\
FORMAT = dest_nt_domain::"$1"

1. The following regex escape the 2nd backslash.
2. Splunk btool won't escape the 2nd backslash.
3. As a result, next attribute, FORMAT, is not parsed as an attribute.

In any case, the regex is not great because it look for any characters again and again then, then check backslash exist.
The following regex should fix the issue and provide better pattern matching for this issue.

[User_ID_as_dest_nt_domain]
SOURCE_KEY = User_ID
REGEX = ^([^\\]+)
FORMAT = dest_nt_domain::$1

[Security_ID_as_dest_nt_domain]
SOURCE_KEY = Security_ID
REGEX = ^([^\\]+)
FORMAT = dest_nt_domain::$1

[Target_Account_ID_as_dest_nt_domain]
SOURCE_KEY = Target_Account_ID
REGEX = ^([^\\]+)
FORMAT = dest_nt_domain::$1

[Security_ID_as_src_nt_domain]
SOURCE_KEY = Security_ID
REGEX = ^([^\\]+)
FORMAT = src_nt_domain::$1

Any additional suggestion or advice would be appreciated.

[Masa]

Note: Splunk_TA_windows v4.8.0 was released. The same issue still exist.

[krdo]

We have the same problem; When you look at the [Security_ID_as_src_nt_domain] in transforms.conf it seems like the two backslashes at the end of the line REGEX = (.+)\\ cause the problem. They seem to escape the following new line character and therefore the line FORMAT = src_nt_domain::"$1" is appended to the REGEX line. You can verify this by navigating to Splunk Web > Settings > Field transformations > Security_ID_as_src_nt_domain.

[jcoates_splunk]

agreed, I think that's the issue -- filed a bug.