All Apps and Add-ons

Automatic backslash-escape in Checkboxes module (Sideview Utils 3.0)

rkursawe
Explorer

I would like to use the Checkboxes module to choose imported error messages. Sometimes this values contains characters which should be escaped to proceed it in the following search query. I understood the previous answer so that the module itself manages this escaping. But this does only work with a backslash. Other characters like quotations marks ("), box brackets ([) will not be escaped.

For example:

C:\Program Files -> C:\\Program Files

but

name="value" -> name="value" instead of name=\"value\"

Because of the backslash escaping there is no workarround possible. \" will be \\" after passing the module. Is this a bug in the module?

My source code is like the following. The first search module fills the Checkboxes and the second search uses "$excludeevents$" to exclude chosen checkbox values.

<module name="Search" ...
...
  <module name="Checkboxes" >
  <param name="name">excludeevents</param>
  <param name="valueField">Errordetail</param>
  <param name="labelField">Errordetail</param>
  <param name="template"> NOT like(Errordetail,"%$value$%") </param>
  <param name="separator">+ AND +</param>
  <param name="outerTemplate">| where $value$ </param>
    <module name="Search" ...
    ...
    </module>
  </module>
</module>

The solution in general works fine. There is only a problem with characters which have to be escaped to use it in a Splunk search query.

Thanks in advance!

sideview
SplunkTrust
SplunkTrust

This is a very longstanding bug with the Sideview form element modules, around double-quote handling. Very few have run into it, and in the past, in the cases I've seen at least, we've been able to find other workarounds.

I believe after some fresh thinking this morning that I have a fix for it and I am beginning to put that fix through its paces. The fix only changes a few lines of code, fixes the issue across not just Checkboxes but also Pulldown, Radio, Tabs and ArrayValueSetter, and I don't think it has any side effects. However my paranoia level is very high about it because the set of all use cases for all these modules is a big set and I can't cause any regressions. Thus I need to write a lot more testcases and let it burn in with some of our apps for a while before I'll release it.

Some background.

The history of search-language escaping in general in Splunk UI modules is a pretty messy one. The Sideview modules have improved things greatly over the last 3 years, but at considerable investment of time and energy. Even so gaps remain, both known and unknown. The gaps have at least steadily closed over the years, and bugs have been fixed rapidly once reported publicly. You can actually search through the Sideview Utils release notes and see for yourself that there have been a steady diet of improvements around escaping behavior. http://sideviewapps.com/apps/sideview-utils/release-notes/

It at first seems like form element modules should always automatically escape their selected values to "work" in the search language, but the truth is a lot messier. For instance Textfield and Table drilldowns are very commonly used to generate $foo$ tokens that are already complete search expressions for direct substitution. Such expressions need to back backslash-escaped but doing any further escaping on these values is a disaster. Yet still we must automatically escape dynamic options in our Pulldowns, allow things to be templatized into quoted expressions, and in the middle find somewhere to draw what seem like so many lines in the sand. (Things are far far better now than they were in the early days, and the lines have sensible use cases and comments and contracts etc. 😃

HOWEVER. The larger trickier big picture is beside the point. The problem you have found here can be defined very narrowly - concerning only Sideview modules that have "template" params, where the user is templating the selected value inside the template.

In these modules the problem is well defined and a well defined contract can be made - when the module "templatizes" the value, it can look at the template, check whether the value is going inside a quoted expression in the template string. If so, it escapes quote chars in the templated value. So... that's the change I want to put in, and since at this point 3.0.2 is looking to have a lot of improvements in it already, it'll be 3.1, out in a week or two.

Thank you VERY much for writing this up and helping me get a fresh look on a very well-worn topic.

sideview
SplunkTrust
SplunkTrust

Excellent. I'm glad to hear it. 😃

0 Karma

rkursawe
Explorer

I downloaded the new version and it works very well. The ".rawValue" part avoids double escaping and the checkboxes module escapes correctly. Thank you very much!

0 Karma

sideview
SplunkTrust
SplunkTrust

As to making the escaping configurable, I've thought about that a lot but it seems like a "worst-practice" direction for the framework that'll create a lot of tangled use cases with just as much surface area for bugs. or at least just as much surface area for "sideview support cases". I'm still trying to prevent 99% of developers from thinking about this stuff at all. And as a last resort there's always a customBehavior!

0 Karma

sideview
SplunkTrust
SplunkTrust

Yep. backslashes in Sideview params are generally treated as literal backslash characters in the content, so as such they always get escaped in anything that is destined for the search language, like the main $myNameParam$ tokens. Note however there's always a $myNameParam.rawValue$ token that is to be used for url's, HTML displayed to the user and redirector arguments, and in there you dont get escaped backslashes..

0 Karma

rkursawe
Explorer

The other point is, is it possible to configure the decition about escaping as a module paramter? Then the user itself can decide about escaping and switch it off or on? Or you relocate escaping in a separate module?

Your utilities are very complex and I'm shure, that you have an more profound view on it, but I hope ideas might be helpful. I'm curious about your solution.

0 Karma

rkursawe
Explorer

Its reasonable that side effects are possible. Escaping something can be a never ending story. By the way, I have an combination of Textfield, ValueSetter (to split the term) and an ArrayValueSetter (to build a query part) which alreay brings an double escaping. But a normal pulldown module escapes fine. I totaly understand your concerns.
Regarding the checkboxes module, I'm not shure, but there might be a point in your code which escapes the backslash itself. Every backslash will always escaped by the module, only quotes and pipes are not escaped.

0 Karma

rkursawe
Explorer

\" will not be \" -> (one backslash)" will be (double backslash)" after passing the module. Or with other words, the module escapes backslashes only.

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.