How to troubleshoot eval expressions?

[fabriziorti]

I'm new to Splunk, and I am trying to figure out how the eval command works in searches.
Sometimes I don't get any result, but no errors/warnings are generated (not even on any log files I can see).
How do I troubleshoot an 'eval' that does not produce anything?
For example, I was using a simple expression like:
sourcetype="rti_avdemo" | eval foo=action+1 | table action, foo
where I thought 'action' was a number. That did not produce anything. Then I figured out that 'action' is a multi-value.
In this case, eval does not produce anything, and I get no errors, warnings. How can I get some information that can help me ?

Thanks!
Fab

[fabriziorti]

Thank you to both @DavidHourani and @goodair for your replies. Although I would like to see more explanation why eval is not producing a result, I understand that sometimes an expression just does not evaluate to anything.
No

[DavidHourani]

Hi @fabriziorti,

That's a good question, and I would say people that have used Splunk for a long time have gotten used to this but it is in fact weird to not get errors.

So to answer your question, you will not get errors anywhere for an eval not giving results as this is by design how it is meant to work: "When there are results show them, no results -- ok no problem."

Before writing an eval you just need to be sure that you're hitting the right fields, with the right type and then you're good to go, it would be easier to get an error message for it but for now it is what it is. Feel free to post any questions if you're stuck on an eval expression and you're not getting results the community would be more than happy to help.

Cheers,
David

[goodair]

You'll need to look into your data to be aware of types before writing evals and other commands, otherwise unexpected results like this will occur.

With this particular case regarding the multi-value, you can use the mvexpand command to create a new event for each value in that field:

sourcetype="rti_avdemo" | mvexpand action | eval foo=action+1 | table action, foo

This will split all the mutli-value fields up into separate events (copying the rest of the fields), and then allow you to run your eval against each event.

More information on mvexpand and eval can be found below:
https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Mvexpand
https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/eval

[fabriziorti]

Hmm, thanks for your good points, but "looking at my data" doesn't really answer my question. Errors are made, especially when we start with Splunk. Having some sort of feedback like "cannot apply the '+' operator over a multi-value" would have helped me.

[goodair]

Unfortunately looking at your data is the only solution. In order to parse your data you need to understand what type and format it is in; numbers, strings, multi-values etc.

As far as I know, verbose error messages like that are not applicable to an eval that doesn't evaluate, as the eval itself is syntactically correct. This means the data is either not there (empty field, wrong field name) or the data is not the correct format (string when trying to add numbers.)

Try using commands such as :
| fieldsummary (to get a snapshot of what your data looks like for each field)
| metadata (to get an idea of what types of data is associated with what indexes/sourcetypes/sources)
| eval numberornot = if(isNum(fieldnamehere),1,0) | stats count by numberornot (tells you how many of your events have a number in that fieldnamehere field, if all of them, then you are probably safe to do calculations, if it is a mixture then build case or if statements to cover the possible inputs

[fabriziorti]

Great suggestions @goodair! Thanks