topic Why is multi-value foreach command not returning expected values from template <<ITEM>>? in Splunk Search

Why is multi-value foreach command not returning expected values from template <>?

Splunk_Hatched — Thu, 28 Jul 2022 14:55:13 GMT

Hi,

I have a multi-value field numbers with each of its values in the format of two numbers separated by a comma (for example 52,29). For all of these values, I want to have an if statement that does a comparison on both the first number and second number and then return either "true" or "false".

Currently I have been using the foreach loop with the multi-value mode. However, when debugging why I am receiving the error below, I found that the default template value <<ITEM>> appears to always return null instead of the values of numbers (isnotnull('<<ITEM>>') returns False).

Shown below is how I am trying to extract the leftmost number using regex with replace and then check if it is greater than 5. Is there something wrong with this search?

| foreach mode=multivalue numbers
[| eval results=if(tonumber(replace('<<ITEM>>'), ",\d+", "")) > 5, "true", "false")]

This is the error I get for the search above:

Thanks in advance.

Re: Multi-value foreach command not returning expected values from template <>

bowesmana — Thu, 28 Jul 2022 03:03:15 GMT

You have too many closing brackets - remove the one after ITEM

('<<ITEM>>')

Re: Multi-value foreach command not returning expected values from template <>

yuanliu — Thu, 28 Jul 2022 03:17:32 GMT

First, there's a syntax error: you closed replace() function at '<<ITEM>>', instead of after "".

But a bigger problem is the unclear requirement. For example, if given two values in numbers, if one of them is greater than 5 but another is less than 5, what should result be? The solution will depend on such details.

The following will give you a multivalued results, which may not be what you wanted.

| foreach mode=multivalue numbers [eval results=mvappend(results, if(tonumber(replace('<<ITEM>>', ",\d+", "")) > 5, "true", "false"))]

Suppose numbers is (8, 2), results will be ('true', 'false'); if numbers is (2, 8), results is ('false', 'true')

Re: Multi-value foreach command not returning expected values from template <>

Splunk_Hatched — Thu, 28 Jul 2022 17:42:59 GMT

Hello yuanliu,

Thank you for pointing out the extra parentheses and lack of clarity in my search. Perhaps this adjusted search would help clarify further what I originally intended:

| foreach mode=multivalue numbers [eval results=mvappend(results, if(tonumber(replace('<<ITEM>>', ",\d+", "")) > 5 AND tonumber(replace('<<ITEM>>', "^\d+,", "")) > 5, "true", "false"))]

While this adjusted search is now closer to what I intended, the results field only creates two outputs when I wanted to create three (one for each of the three values in the numbers field). The foreach command appears not to have reached the third value.

Further testing to check what '<<ITEM>>' actually returns appears to give a Null value (see the screenshot of testParseNumbers shown below).

| eval testParseNumbers="" | foreach mode=multivalue numbers [eval results=mvappend(results, if(tonumber(replace('<<ITEM>>', ",\d+", "")) > 5 AND tonumber(replace('<<ITEM>>', "^\d+,", "")) > 5, "true", "false")), testParseNumbers=testParseNumbers.tostring('<<ITEM>>')]

I would appreciate suggestions on how to fix this. Thank you.

Re: Multi-value foreach command not returning expected values from template <>

Splunk_Hatched — Thu, 28 Jul 2022 17:46:21 GMT

Thank you for clearly pointing that out. I would appreciate further feedback about my query in my reply to yuanliu

Re: Multi-value foreach command not returning expected values from template <>

yuanliu — Fri, 29 Jul 2022 02:54:55 GMT

The screenshot of numbers shows that your values can be negative, which is represented by "-", not matching "\d+". (It really helps to illustrate data in text, especially as yours are already anonymized.) The following should work on that data set

| foreach mode=multivalue numbers [eval results=mvappend(results, if(tonumber(replace('<<ITEM>>', ",[-\d]+", "")) > 5 AND tonumber(replace('<<ITEM>>', "^[-\d]+,", "")) > 5, "true", "false"))]

Using the sample data shown in that screenshot, the output is

numbers	results
-1,-1 0,25535 22,3389	false false true

Is this what you expect? Again, it is perhaps easier to verbalize your criteria, assisted by code or pseudo-code. I interpret your intention as follows: given multivalue pairs of comma-delimited integers, calculate multivalue results; corresponding to each pair, return true if both numbers are greater than 5, false otherwise. (I kind of made this wordier than necessary, but that's the idea.)

I have some suspicion, though, that a multivalue field of results may not be what you wanted, as it may complicate subsequent processing.

Re: Multi-value foreach command not returning expected values from template <>

Splunk_Hatched — Fri, 29 Jul 2022 20:49:33 GMT

Hi yuanliu,

Thank you for pointing out my mistakes in the regex and lack of clarity in my desired criteria. Your description of my intended output for results are correct.
You are also correct that a multi-value field is not optimal for my desired results field. Thinking this over, I resolved the issue by using mvexpand on numbers before applying the Boolean logic (i.e. expression>5).

I greatly appreciate your help to troubleshoot through both my SPL expressions and the implementation of the search.