Issue making value lowercase through eval in props.conf and or transforms.conf

[fatsug]

Trying to make som CIM compliant aliases as well as making the values standard format for compliance with another CIM compliant app I have run into a wall.

In my TA I have the following line in my props.conf

FIELDALIAS-severity = plugin.pluginattributes.risk_information.risk_factor AS severity

Values in plugin.pluginattributes.risk_information.risk_factor ar Capitized and because of this also severity. In order to NOT have to rewrite a bunch of macros and searches in the other app I would need the values in severity to be all lowercase.

Initially I placed an eval in my props.conf file

EVAL-severity = lower(severity)

In my mind this should have fixed the problem, but it had no effect. I tried using the actual field name

EVAL-plugin.pluginattributes.risk_information.risk_factor = lower(plugin.pluginattributes.risk_information.risk_factor)

With no luck. After a bit of search online I tried using a transfom by adding a 

TRANSFORMS-lowerseverity = lower_severity

to my props.conf and an eval to transforms,

[lower_severity]
EVAL-severity = lower(severity)

but still no effect.

This is basic JSON log being collected, fields are extracted at search time. Can I solve this issue using props or a combination of props and transforms? Or do I have to use sed/regex if I want to convert Capitilized words into all lowercase? 

All the best

[gcusello · Accepted solution]

Hi @fatsug ,

there's a sequence to execute transformations, please try:

EVAL-severity = lower(plugin.pluginattributes.risk_information.risk_factor)

even if you shouldn't need to do this because mopst searches aren't case sensitive, only few fields (e.g. tag) are case sensitive.

Ciao.

Giuseppe

[fatsug]

Facepalm moment... The initial solution actually worked

props.conf

EVAL-severity = lower(severity)

Only problem was, there was a # comment made inline which I did not include. Removing this and only using the correct eval works like a charm.

Thank you @gcusello!

[gcusello]

Hi @fatsug ,

good for you, see next time!

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated 😉

[fatsug]

Hello again @gcusello 

I may be a bit low on caffein so maybe I'm not seeing this correct.

The suggested solution in my props.conf file is the same solution I initially tried, which did not work. I tried moving the EVAL to my transforms.conf with no luck.

The SPL search

<base search>
| eval severity = lower(severity)
| stats count by severity

Works just fine, I get lower case values just as expected. But I cannot get this to work in props.conf and/or transforms.conf.

Asking the all-knowing AI it tries to tell me that EVAL lower in props.conf only works on indexed fields. So am I doing or reading something incorrectly, or is there some other issue I am missing?

[gcusello]

Hi @fatsug ,

no props.conf doesn't run only on indexed fields, it works on all fields, but there's a sequence in executions.

If you put the eval in the search, it runs on an already done transformation for the severity field, so it correctly runs.

If you put it in props.conf, you must analyze the sequence of transformations, in you your case, you have a rename that probably is executed after the eval transformation, for this reason the severity field isn't still available for the eval, and for this reason I hint to put in the eval the name of the severity field before the rename.

Ciao.

Giuseppe

[fatsug]

Hello @gcusello 

No, I am unsure where in all the evals the need arises. I was hoping for a "quick workaround" 🙂

Just to make sure, should this be in the transforms.conf file or should I go back to only the props.conf eval?

[gcusello]

Hi @fatsug ,

I'd modify the calculated field also beacuse it's the easiest way, even if I don't think that the case is relevant.

Ciao.

Giuseppe