topic Re: case_sensitive_match = false does not appear to work in Splunk Search

case_sensitive_match = false does not appear to work

kmattern — Tue, 29 Sep 2020 12:40:48 GMT

I have at least one lookup table where a field may be upper, lower or mixed case but the search value, pulled from log data to do the lookup, may not match the case. For example the log record may have:

cs_username = jk_midstate01

While the lookup table may have that field as:

cs_username = jk_MidState01

And it cannot be changed because the field is referenced in multiple places by different searches. It is not possible to change the case of either field. I put the following in etc\system\local\transforms.conf and restarted Splunk. At the same time I ensured there was no other place where the specific lookup table was referenced with the case_sensitive_match line.

[default]
case_sensitive_match = false

The searches I write to test the case insensitivity always fail, indicating to me that the transforms.conf entry is not valid. Of course the Splunk documentation is rife with meaty examples of using case_sensitive_match.

Any advice?

Thanks

Note: My PC browser does not show all comments and does not give edit capability so I have to use my phone. Forgive my fat fingers.

Re: case_sensitive_match = false does not appear to work

gcusello — Tue, 31 Jan 2017 14:57:51 GMT

Hi kmattern,
I found this problem in host detection and I solved it transforming always in uppercase (or lowercase)

 | eval host=upper(host)

Bye.
Giuseppe

Re: case_sensitive_match = false does not appear to work

somesoni2 — Tue, 31 Jan 2017 14:59:20 GMT

Is there typo in either values? In the log record value isjk_midstate01 and in lookup you mentioned jk_Mid_State01 which is different value altogether due to additional underscore.

Re: case_sensitive_match = false does not appear to work

kmattern — Tue, 31 Jan 2017 15:11:26 GMT

That's a typo. It should be 01 in each instance.

Re: case_sensitive_match = false does not appear to work

kmattern — Tue, 31 Jan 2017 15:12:30 GMT

I cannot force the case because I have no idea what the case actually is in the lookup.

Re: case_sensitive_match = false does not appear to work

gcusello — Tue, 31 Jan 2017 15:16:48 GMT

if you have to search, you can transforms in uppercase both your search results and lookup

your_search
| eval cs_username=upper(cs_username)
| search [| inputlookup your_lookup.csv 
   | eval cs_username=upper(cs_username)
   | fields cs_username
   ]
...

Bye.
Giuseppe

Re: case_sensitive_match = false does not appear to work

somesoni2 — Tue, 31 Jan 2017 15:25:07 GMT

Only one underscore right? Also, does the lookup being used to enrich the data (add additional fields from lookoup) or for filter?

Re: case_sensitive_match = false does not appear to work

kmattern — Tue, 31 Jan 2017 15:38:32 GMT

I would still like an answer to my original question, how do I get transforms.conf to work?

Re: case_sensitive_match = false does not appear to work

kmattern — Tue, 29 Sep 2020 12:38:12 GMT

I haven't tried a sub search. If I absolutely have to I will but I'd still like to get case_sensitive_match=false to work.

Re: case_sensitive_match = false does not appear to work

somesoni2 — Tue, 31 Jan 2017 15:50:52 GMT

Can you update your question to fix the typo? The location of the transforms.conf you update in the question is etc\system\trsansforms.conf, shouldn't that be etc/system/local/transforms.conf ? Also, why update the default stanza, why not for specific lookup definition (assuming you've already setup a lookup definition for your lookup table. These additional setting works only on lookup definitions)?

Re: case_sensitive_match = false does not appear to work

kmattern — Tue, 29 Sep 2020 12:38:15 GMT

It should be jk_midstate01 and jk_MidState01 in the example above. I was trying to attend a boring phone con while writing the question. sorry.

Re: case_sensitive_match = false does not appear to work

somesoni2 — Tue, 31 Jan 2017 16:05:52 GMT

Ok.. I've made the changes to question to reflect that. Are you using lookup definition for your CSV lookup table file? Can you try adding case_sensitive_match = false under the transforms.conf entry for that lookup definition?

Re: case_sensitive_match = false does not appear to work

kmattern — Tue, 31 Jan 2017 16:17:07 GMT

It is updated.

Re: case_sensitive_match = false does not appear to work

kmattern — Tue, 31 Jan 2017 16:22:50 GMT

That's where I had it originally, no joy.

Re: case_sensitive_match = false does not appear to work

woodcock — Sat, 04 Mar 2017 05:23:45 GMT

Do not use the [default] header stanza. Instead use your lookup name as the stanza header instead. I have done this dozens of times and never had a problem. I have never tried this setting in the [default] stanza.