"Bad regex" warning after upgrading from Enterprise 6.5.1 to 7.0.1.

[kentcoble]

After upgrading Splunk on a test server from 6.5.1 to 7.0.1, we receive the following message when starting Splunk:

01-18-2018 17:22:55.079 WARN  btool-support - Bad regex value: '(?msi)(Account\s+Domain\:.*?(Account\s+Domain\:)|Account\s+Domain\:)(?!\s+(\r|\n))\s+(?<dest_nt_domain>[a-zA-Z0-9._[\S-\S][^\r|\n]+)', of param: props.conf / [WinEventLog:Security] / EXTRACT-ms_ad_obj_dest_nt_domain_Account_Domain; why: invalid range in character class
01-18-2018 17:22:55.079 WARN  btool-support - Bad regex value: '(?msi)(Group\s+Domain\:.*?(Group\s+Domain\:)|Group\s+Domain\:)(?!\s+(\r|\n))\s+(?<dest_nt_domain>[a-zA-Z0-9._[\S-\S][^\r|\n]+)', of param: props.conf / [WinEventLog:Security] / EXTRACT-ms_ad_obj_dest_nt_domain_Group_Domain; why: invalid range in character class
01-18-2018 17:22:55.083 WARN  btool-support - Bad regex value: '(?msi)Group\:(\s|\r|\n)(.*?Group\sName\:\s+(?<group_name>[\S-\S][^(\r|\n)]+)(\r|\n))(.*?Group\sDomain\:\s+(?<group_domain>[\S-\S][^(\r|\n)]+)(\r|\n))', of param: props.conf / [WinEventLog:Security] / EXTRACT-ms_ad_obj_group_name_domain; why: invalid range in character class
01-18-2018 17:22:55.083 WARN  btool-support - Bad regex value: '(?msi)Member\:(\s|\r|\n)(.*?Account\sName\:\s+(?<member_dn>[\S-\S][^(\r|\n)]+)(\r|\n))', of param: props.conf / [WinEventLog:Security] / EXTRACT-ms_ad_obj_member_dn_Account_Name; why: invalid range in character class
01-18-2018 17:22:55.083 WARN  btool-support - Bad regex value: '(?msi)(New\sGroup\:|Group\:)(\s|\r|\n)(.*?Security ID:(\s+(?<group_domain>[^\x5C{1}]+)\x5C{1}|\s+)(?<group_id>[\S-\S][^(\r|\n)]+)(\r|\n))', of param: props.conf / [WinEventLog:Security] / EXTRACT-ms_ad_obj_member_group_name_id_domain; why: invalid range in character class
01-18-2018 17:22:55.083 WARN  btool-support - Bad regex value: '(?msi)Member\:(\s|\r|\n)(.*?Security ID:(\s+(?<member_domain>[^\x5C{1}]+)\x5C{1}|\s+)(?<member_id>[\S-\S][^(\r|\n)]+)(\r|\n))', of param: props.conf / [WinEventLog:Security] / EXTRACT-ms_ad_obj_member_id_member_domain_Security_ID; why: invalid range in character class
01-18-2018 17:22:55.086 WARN  btool-support - Bad regex value: '(?msi)(Logon\s+ID\:.*?(Logon\s+ID\:)|Logon\s+ID\:)(?!\s+(\r|\n))\s+(?<session_id>[a-zA-Z0-9._[\S-\S][^\r|\n]+)', of param: props.conf / [WinEventLog:Security] / EXTRACT-ms_ad_obj_session_id; why: invalid range in character class
01-18-2018 17:22:55.087 WARN  btool-support - Bad regex value: '(?msi)(?:Account\s+Domain\:(?!\s+(\r|\n))\s+(?<src_nt_domain>[a-zA-Z0-9._[\S-\S][^\r|\n]+))', of param: props.conf / [WinEventLog:Security] / EXTRACT-ms_ad_obj_src_nt_domain_Account_Domain; why: invalid range in character class
01-18-2018 17:22:55.087 WARN  btool-support - Bad regex value: '(?msi)(?:Account\s+Name\:(?!\s+(\r|\n))\s+(?<src_user>[a-zA-Z0-9._[\S-\S][^\r|\n]+))', of param: props.conf / [WinEventLog:Security] / EXTRACT-ms_ad_obj_src_user_Account_Name; why: invalid range in character class
01-18-2018 17:22:55.087 WARN  btool-support - Bad regex value: '(?msi)(Account\s+Name\:.*?(Account\s+Name\:)|Account\s+Name\:)(?!\s+(\r|\n))\s+(?<user>[a-zA-Z0-9._[\S-\S][^\r|\n]+)', of param: props.conf / [WinEventLog:Security] / EXTRACT-ms_ad_obj_user_Account_Name; why: invalid range in character class

This does not prevent Splunk from running, however we're concerned about any potential impact on parsing logs.

If it helps, we have the following add-ons installed:

• Splunk Add-On for Windows Infrastructure 1.4.1
• Splunk Add-On for Microsoft Windows 4.8.4
• Splunk Add-On for Microsoft Active Directory 1.0.0
• Splunk Add-On for Microsoft Windows DNS 1.0.1
• MS Windows AD Objects 3.1.1

When I run a grep "WinEventLog:Security" /opt/splunk/etc/apps/*/default/* I cannot find any matches in a props.conf file, so I'm a little confused as to where this is being found.

Any suggestions?

[agadayev · Accepted solution]

Kudos to "kentcoble". I am in no way an expert in this but I went to investigate a bit further using regex101.com. His suspicion of [\S-\S] was correct adding another backslash() on the right sight of the first "S" does the same thing and gets rid of the error. In detail: go to "\etc\apps\ms_windows_ad_objects\default" copy the props.conf file to "\etc\apps\ms_windows_ad_objects\local" and in the local copy of props.conf for every line in the error find [\S-\S] and change it to [\S\-\S]. This fixed my issue.

[kentcoble]

You mean change [\S-\S] to [\S\-\S]?

[jspears]

We ran into this as well. It looks like Splunk 7 has stricter interpretation of the character class construct, so [\S-\S] means "non-whitespace to non-whitespace" instead of "non-whitespace dash non-whitespace." Ultimately this makes the behavior more predictable.

[agadayev]

Yes exactly, I went to correct my mistake but saw that I wrote it correctly but it displays incorrect. How do I wright it in red the way you do?

[kentcoble]

That's a code block/snippet. You'll want to encapsulate the text with backticks (). Perhaps you could use two backslashes instead, ex.[\S\-\S]`. As a test: [\S\-\S]

[agadayev]

Great thanks, learned something new. Two backslashes did the trick.

[Azeemering]

Looks like the error stems from an invalid regular expression, so you'd need to make sure any special characters are appropriately escaped before using them in the regular expression.
You probably need to escape the underscores in the regex string for that extraction.

[kentcoble]

Just tried to escape the underscores to no avail. Some of the strings don't have them in a range (see line 6), however they all seem to contain [\S-\S]. I've never seen this used before in Regex and question what it's suppose to match (range of non-whitespace to non-whitespace??)

[maciep]

not sure about the error itself, but maybe btool or find would be a better way to see which props it's coming from ... although i'd guess the active directory app

splunk btool props list --debug | grep -i EXTRACT-ms_ad_obj

OR

cd $SPLUNK_HOME
find etc/apps -name props.conf | xargs grep -il EXTRACT-ms_ad_obj

[kentcoble]

Thanks for that! Using the first command, it appears that these are coming from ms_windows_ad_objects/default/props.conf. I'll copy the file to the local folder and modify it there for testing.