Splunk changelog for modifications in rex command

[FloSwiip]

Hello,

Is there a place, that ignore, where it is possible to read what has been changed between splunk releases for a particular SPL command like rex ?

I have recently done the migration from version 6.6.4 to 7.0.2 and realised after some time that one of our dashboard based on a custom datamodel was reporting everywhere the following error (repeated with the name of all the indexers at the end) :

The search for datamodel 'mydatamodelname' failed to parse, cannot use search for missing acceleration data

I had to disable the acceleration for it, in order to get from the pivot what was the real error behind that : an extracted field using a rex :

Error in 'rex' command: Encountered the following error while compiling the regex 'cn=[\w- ]+,ou=(?<ldap_myvalue>\w+).+,ou=people,dc=mysite,dc=com': Regex: invalid range in character class 

The conclusion is that it was the part with the [\w- ] that was now generating the error
and just fixed it by putting a back-slash before the dash [\w- ]

But I don't see anything to confirm the change of that in a "What's new" and as it seems that it was not fixed as a bug so nothing also in 'fixed issues'
No comment in the bottom of the http://docs.splunk.com/Documentation/Splunk/7.0.2/SearchReference/Rex

Thank you

[Anam]

Hi Floswiip

Thank you for posting your concern. There is an option at the bottom of the documentation under "Was this topic useful?" for feedback so you can post it there as well. The documentation team is really good about responding to all feedback on any topic. Make sure to share your email so they can get in touch with you for further information.

Thanks!

[493669]

Hi @FloSwiip,
It seems the error was not due to -has not escaped using back slash but due to space after -
so if try same regex with removed space then it will run (without any backslash):

rex "cn=[\w-]+,ou=(?<ldap_myvalue>\w+)"

[FrankVl]

But I guess he intended to include the space as one of the allowed characters. \s might be a better way to do that, but a simple space would also work.

And regardless of what the exact issue is, the point he is making, is that his regex worked fine in 6.6.4, but no longer in 7.0.2 and he is wondering whether the change that is causing that new behaviour was documented somewhere.

Also regex101 for instance accepts his original regex just fine:
https://regex101.com/r/zNhnnu/1

[493669]

as I have tested that - works (without any backslash) in 7.X version
and regarding error I tried a query :

|makeresults|eval a=" asdf-ghg-21"|rex field=a "(?<v>[\w- ])"

it gives me same error in 7.X version

[FrankVl]

You can basically interpret [\w- ] in two ways:

1. the set of characters consisting of: \w, dash and space (which is how 6.6.4 interpreted it and how regex101 interprets it)
2. the range of characters starting at \w until space (which is how 7.0.2 interprets it and obviously results in that error, because \w until space does not make any sense)

I bet you'd get the same result if you wrote [\w-a] or [\w-\d] or something similarly silly (silly based on 7.0.2's interpretation). The range interpretation of x-y only works for stuff like a-z or 0-9. Apparently in the past (and in regex101), for any x-y notation that is not a valid range the dash character is automatically interpreted as the literal dash character.

[FloSwiip]

Hi, thank you for all your answers.

To give more context about that :
The rex pattern was not written by me that is why I did not commented the way it was done.
The author committed it in January, the dashboard was review and approved, then I did splunk upgrade last week, and today I was warned by the owner that he had error everywhere when he tried to present its dashboard to someone else. Oupps, I though first about permissions issue, until I did what I explained in the first post.

Ok I was lazy enough to build a 6.6.4 and put the dashboard, the datamodel and some ldap data, I tried looking at the doc first 🙂
Now I just did that test and yes confirm that rex changed between 6.5.x and 7.0.x as you explained

I still think it would be good to know if some other changes were included, so I will follow asiddique_splunk commendation, and report here there is something interesting to share.