Installation

"Bad regex" warning after upgrading from Enterprise 6.5.1 to 7.0.1.

kentcoble
Explorer

After upgrading Splunk on a test server from 6.5.1 to 7.0.1, we receive the following message when starting Splunk:

01-18-2018 17:22:55.079 WARN  btool-support - Bad regex value: '(?msi)(Account\s+Domain\:.*?(Account\s+Domain\:)|Account\s+Domain\:)(?!\s+(\r|\n))\s+(?<dest_nt_domain>[a-zA-Z0-9._[\S-\S][^\r|\n]+)', of param: props.conf / [WinEventLog:Security] / EXTRACT-ms_ad_obj_dest_nt_domain_Account_Domain; why: invalid range in character class
01-18-2018 17:22:55.079 WARN  btool-support - Bad regex value: '(?msi)(Group\s+Domain\:.*?(Group\s+Domain\:)|Group\s+Domain\:)(?!\s+(\r|\n))\s+(?<dest_nt_domain>[a-zA-Z0-9._[\S-\S][^\r|\n]+)', of param: props.conf / [WinEventLog:Security] / EXTRACT-ms_ad_obj_dest_nt_domain_Group_Domain; why: invalid range in character class
01-18-2018 17:22:55.083 WARN  btool-support - Bad regex value: '(?msi)Group\:(\s|\r|\n)(.*?Group\sName\:\s+(?<group_name>[\S-\S][^(\r|\n)]+)(\r|\n))(.*?Group\sDomain\:\s+(?<group_domain>[\S-\S][^(\r|\n)]+)(\r|\n))', of param: props.conf / [WinEventLog:Security] / EXTRACT-ms_ad_obj_group_name_domain; why: invalid range in character class
01-18-2018 17:22:55.083 WARN  btool-support - Bad regex value: '(?msi)Member\:(\s|\r|\n)(.*?Account\sName\:\s+(?<member_dn>[\S-\S][^(\r|\n)]+)(\r|\n))', of param: props.conf / [WinEventLog:Security] / EXTRACT-ms_ad_obj_member_dn_Account_Name; why: invalid range in character class
01-18-2018 17:22:55.083 WARN  btool-support - Bad regex value: '(?msi)(New\sGroup\:|Group\:)(\s|\r|\n)(.*?Security ID:(\s+(?<group_domain>[^\x5C{1}]+)\x5C{1}|\s+)(?<group_id>[\S-\S][^(\r|\n)]+)(\r|\n))', of param: props.conf / [WinEventLog:Security] / EXTRACT-ms_ad_obj_member_group_name_id_domain; why: invalid range in character class
01-18-2018 17:22:55.083 WARN  btool-support - Bad regex value: '(?msi)Member\:(\s|\r|\n)(.*?Security ID:(\s+(?<member_domain>[^\x5C{1}]+)\x5C{1}|\s+)(?<member_id>[\S-\S][^(\r|\n)]+)(\r|\n))', of param: props.conf / [WinEventLog:Security] / EXTRACT-ms_ad_obj_member_id_member_domain_Security_ID; why: invalid range in character class
01-18-2018 17:22:55.086 WARN  btool-support - Bad regex value: '(?msi)(Logon\s+ID\:.*?(Logon\s+ID\:)|Logon\s+ID\:)(?!\s+(\r|\n))\s+(?<session_id>[a-zA-Z0-9._[\S-\S][^\r|\n]+)', of param: props.conf / [WinEventLog:Security] / EXTRACT-ms_ad_obj_session_id; why: invalid range in character class
01-18-2018 17:22:55.087 WARN  btool-support - Bad regex value: '(?msi)(?:Account\s+Domain\:(?!\s+(\r|\n))\s+(?<src_nt_domain>[a-zA-Z0-9._[\S-\S][^\r|\n]+))', of param: props.conf / [WinEventLog:Security] / EXTRACT-ms_ad_obj_src_nt_domain_Account_Domain; why: invalid range in character class
01-18-2018 17:22:55.087 WARN  btool-support - Bad regex value: '(?msi)(?:Account\s+Name\:(?!\s+(\r|\n))\s+(?<src_user>[a-zA-Z0-9._[\S-\S][^\r|\n]+))', of param: props.conf / [WinEventLog:Security] / EXTRACT-ms_ad_obj_src_user_Account_Name; why: invalid range in character class
01-18-2018 17:22:55.087 WARN  btool-support - Bad regex value: '(?msi)(Account\s+Name\:.*?(Account\s+Name\:)|Account\s+Name\:)(?!\s+(\r|\n))\s+(?<user>[a-zA-Z0-9._[\S-\S][^\r|\n]+)', of param: props.conf / [WinEventLog:Security] / EXTRACT-ms_ad_obj_user_Account_Name; why: invalid range in character class

This does not prevent Splunk from running, however we're concerned about any potential impact on parsing logs.

If it helps, we have the following add-ons installed:

  • Splunk Add-On for Windows Infrastructure 1.4.1
  • Splunk Add-On for Microsoft Windows 4.8.4
  • Splunk Add-On for Microsoft Active Directory 1.0.0
  • Splunk Add-On for Microsoft Windows DNS 1.0.1
  • MS Windows AD Objects 3.1.1

When I run a grep "WinEventLog:Security" /opt/splunk/etc/apps/*/default/* I cannot find any matches in a props.conf file, so I'm a little confused as to where this is being found.

Any suggestions?

Labels (3)
1 Solution

agadayev
Path Finder

Kudos to "kentcoble". I am in no way an expert in this but I went to investigate a bit further using regex101.com. His suspicion of [\S-\S] was correct adding another backslash() on the right sight of the first "S" does the same thing and gets rid of the error. In detail: go to "\etc\apps\ms_windows_ad_objects\default" copy the props.conf file to "\etc\apps\ms_windows_ad_objects\local" and in the local copy of props.conf for every line in the error find [\S-\S] and change it to [\S\-\S]. This fixed my issue.

View solution in original post

agadayev
Path Finder

Kudos to "kentcoble". I am in no way an expert in this but I went to investigate a bit further using regex101.com. His suspicion of [\S-\S] was correct adding another backslash() on the right sight of the first "S" does the same thing and gets rid of the error. In detail: go to "\etc\apps\ms_windows_ad_objects\default" copy the props.conf file to "\etc\apps\ms_windows_ad_objects\local" and in the local copy of props.conf for every line in the error find [\S-\S] and change it to [\S\-\S]. This fixed my issue.

View solution in original post

kentcoble
Explorer

You mean change [\S-\S] to [\S\-\S]?

0 Karma

jspears
Communicator

We ran into this as well. It looks like Splunk 7 has stricter interpretation of the character class construct, so [\S-\S] means "non-whitespace to non-whitespace" instead of "non-whitespace dash non-whitespace." Ultimately this makes the behavior more predictable.

0 Karma

agadayev
Path Finder

Yes exactly, I went to correct my mistake but saw that I wrote it correctly but it displays incorrect. How do I wright it in red the way you do?

0 Karma

kentcoble
Explorer

That's a code block/snippet. You'll want to encapsulate the text with backticks (). Perhaps you could use two backslashes instead, ex.[\S\-\S]`. As a test: [\S\-\S]

0 Karma

agadayev
Path Finder

Great thanks, learned something new. Two backslashes did the trick.

0 Karma

Azeemering
Builder

Looks like the error stems from an invalid regular expression, so you'd need to make sure any special characters are appropriately escaped before using them in the regular expression.
You probably need to escape the underscores in the regex string for that extraction.

0 Karma

kentcoble
Explorer

Just tried to escape the underscores to no avail. Some of the strings don't have them in a range (see line 6), however they all seem to contain [\S-\S]. I've never seen this used before in Regex and question what it's suppose to match (range of non-whitespace to non-whitespace??)

0 Karma

maciep
Champion

not sure about the error itself, but maybe btool or find would be a better way to see which props it's coming from ... although i'd guess the active directory app

splunk btool props list --debug | grep -i EXTRACT-ms_ad_obj

OR

cd $SPLUNK_HOME
find etc/apps -name props.conf | xargs grep -il EXTRACT-ms_ad_obj

0 Karma

kentcoble
Explorer

Thanks for that! Using the first command, it appears that these are coming from ms_windows_ad_objects/default/props.conf. I'll copy the file to the local folder and modify it there for testing.

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!