Splunk Search

case_sensitive_match = false does not appear to work

Builder

I have at least one lookup table where a field may be upper, lower or mixed case but the search value, pulled from log data to do the lookup, may not match the case. For example the log record may have:

csusername = jkmidstate01

While the lookup table may have that field as:

csusername = jkMidState01

And it cannot be changed because the field is referenced in multiple places by different searches. It is not possible to change the case of either field. I put the following in etc\system\local\transforms.conf and restarted Splunk. At the same time I ensured there was no other place where the specific lookup table was referenced with the casesensitivematch line.

[default]
casesensitivematch = false

The searches I write to test the case insensitivity always fail, indicating to me that the transforms.conf entry is not valid. Of course the Splunk documentation is rife with meaty examples of using casesensitivematch.

Any advice?

Thanks

Note: My PC browser does not show all comments and does not give edit capability so I have to use my phone. Forgive my fat fingers.

0 Karma

Esteemed Legend

Do not use the [default] header stanza. Instead use your lookup name as the stanza header instead. I have done this dozens of times and never had a problem. I have never tried this setting in the [default] stanza.

0 Karma

Builder

That's where I had it originally, no joy.

0 Karma

Builder

It is updated.

0 Karma

Builder

It should be jkmidstate01 and jkMidState01 in the example above. I was trying to attend a boring phone con while writing the question. sorry.

0 Karma

SplunkTrust
SplunkTrust

Ok.. I've made the changes to question to reflect that. Are you using lookup definition for your CSV lookup table file? Can you try adding case_sensitive_match = false under the transforms.conf entry for that lookup definition?

0 Karma

Builder

I would still like an answer to my original question, how do I get transforms.conf to work?

0 Karma

SplunkTrust
SplunkTrust

Can you update your question to fix the typo? The location of the transforms.conf you update in the question is etc\system\trsansforms.conf, shouldn't that be etc/system/local/transforms.conf ? Also, why update the default stanza, why not for specific lookup definition (assuming you've already setup a lookup definition for your lookup table. These additional setting works only on lookup definitions)?

0 Karma

Builder

That's a typo. It should be 01 in each instance.

0 Karma

SplunkTrust
SplunkTrust

Only one underscore right? Also, does the lookup being used to enrich the data (add additional fields from lookoup) or for filter?

0 Karma

SplunkTrust
SplunkTrust

Is there typo in either values? In the log record value isjk_midstate01 and in lookup you mentioned jk_Mid_State01 which is different value altogether due to additional underscore.

0 Karma

Legend

Hi kmattern,
I found this problem in host detection and I solved it transforming always in uppercase (or lowercase)

 | eval host=upper(host)

Bye.
Giuseppe

0 Karma

Builder

I haven't tried a sub search. If I absolutely have to I will but I'd still like to get casesensitivematch=false to work.

0 Karma

Builder

I cannot force the case because I have no idea what the case actually is in the lookup.

0 Karma

Legend

if you have to search, you can transforms in uppercase both your search results and lookup

your_search
| eval cs_username=upper(cs_username)
| search [| inputlookup your_lookup.csv 
   | eval cs_username=upper(cs_username)
   | fields cs_username
   ]
...

Bye.
Giuseppe

0 Karma