topic Re: how to search in array inside a foreach loop in Splunk Search

how to search in array inside a foreach loop

mrsampson — Tue, 14 Jan 2025 18:22:21 GMT

This is an example of the structure of my data and the query I am currently using. I have tried around 10 different solutions based on various examples from stackoverflow.com and community.splunk.com. But I have not figured out how to change this query such that eval Tag = "Tag1" can become an array eval Tags = ["Tag1", "Tag4"] and I will get entries for all tags that exist in the array. Could someone guide me in the right direction?

| makeresults | eval _raw = "{ \"Info\": { \"Apps\": { \"ReportingServices\": { \"ReportTags\": [ \"Tag1\" ], \"UserTags\": [ \"Tag2\", \"Tag3\" ] }, \"MessageQueue\": { \"ReportTags\": [ \"Tag1\", \"Tag4\" ], \"UserTags\": [ \"Tag3\", \"Tag4\", \"Tag5\" ] }, \"Frontend\": { \"ClientTags\": [ \"Tag12\", \"Tag47\" ] } } } }" | eval Tag = "Tag1" | spath | foreach *ReportTags{} [| eval tags=mvappend(tags, if(lower('<<FIELD>>') = lower(Tag), "<<FIELD>>", null()))] | dedup tags | stats values(tags)

Re: how to search in array inside a foreach loop

ITWhisperer — Tue, 14 Jan 2025 19:08:08 GMT

What does your expected output look like?

Re: how to search in array inside a foreach loop

yuanliu — Wed, 15 Jan 2025 08:34:42 GMT

Do you mean this?

| makeresults | eval _raw = "{ \"Info\": { \"Apps\": { \"ReportingServices\": { \"ReportTags\": [ \"Tag1\" ], \"UserTags\": [ \"Tag2\", \"Tag3\" ] }, \"MessageQueue\": { \"ReportTags\": [ \"Tag1\", \"Tag4\" ], \"UserTags\": [ \"Tag3\", \"Tag4\", \"Tag5\" ] }, \"Frontend\": { \"ClientTags\": [ \"Tag12\", \"Tag47\" ] } } } }" | eval Tag = "Tag1" | spath | foreach *ReportTags{} [| eval tags=mvappend(tags, if(lower('<<FIELD>>') = lower(Tag), '<<FIELD>>', null()))] | dedup tags | stats values(tags)

This gives

values(tags)

Tag1

Tag4

Note when you use double quote on the right-hand side of an eval expression, in quoted entity is used as literal, therefore your original search gives

values(tags)

Info.Apps.MessageQueue.ReportTags{}

Info.Apps.ReportingServices.ReportTags{}

Re: how to search in array inside a foreach loop

mrsampson — Wed, 15 Jan 2025 10:19:51 GMT

for this

| eval Tags = ["Tag3", "Tag4] | spath | foreach *Tags{} [| eval tags=mvappend(tags, if(lower('<<FIELD>>') = lower(Tag), "<<FIELD>>", null()))] | dedup tags | stats values(tags)

I would like to get

Info.Apps.MessageQueue.ReportTags{} Info.Apps.ReportingServices.ReportTags{} Info.Apps.MessageQueue.UserTags{}

Re: how to search in array inside a foreach loop

mrsampson — Wed, 15 Jan 2025 15:08:48 GMT

not really, the main point in here is that my input to this query, instead of a simple value would be an array. e.g.
current input format:

| eval Tag = "Tag1"

desired input format:

| eval Tags = ["Tag3", "Tag4"]

Re: how to search in array inside a foreach loop

ITWhisperer — Wed, 15 Jan 2025 11:23:27 GMT

Splunk's version of arrays is multivalue field, so if you change you input to a multivalue field, you could do something like this

| eval Tag = split(lower("Tag3,Tag4"),",") | spath | foreach *Tags{} [| eval field="<<FIELD>>" | foreach <<FIELD>> mode=multivalue [| eval tags=if(isnull(tags),if(mvfind(Tag,lower('<<ITEM>>')) >= 0, field, null()),mvappend(tags, if(mvfind(Tag,lower('<<ITEM>>')) >= 0, field, null())))] ] | stats values(tags)

Re: how to search in array inside a foreach loop

mrsampson — Wed, 15 Jan 2025 13:43:00 GMT

@ITWhisperer wrote:
Splunk's version of arrays is multivalue field, so if you change you input to a multivalue field, you could do something like this

Thank you for your response and the example, currently it is returnin 0 results for me. Could it have something to do with my Splunk version? I a

Re: how to search in array inside a foreach loop

mrsampson — Wed, 15 Jan 2025 13:43:42 GMT

m using 8.0.5

Re: how to search in array inside a foreach loop

PickleRick — Wed, 15 Jan 2025 14:07:32 GMT

Yes, foreach mode=multivalue appeared in 9.0.0.

Re: how to search in array inside a foreach loop

mrsampson — Wed, 15 Jan 2025 18:19:51 GMT

Does anyone know how to do this on Splunk v8.0.5?

Re: how to search in array inside a foreach loop

ITWhisperer — Wed, 15 Jan 2025 23:56:52 GMT

Try something like this

| eval Tag = split("Tag3,Tag4",",") | mvexpand Tag | spath | foreach *Tags{} [| eval tags=if(mvfind(lower('<<FIELD>>'), "^".lower(Tag)."$") >= 0,mvappend(tags, "<<FIELD>>"), tags)] | stats values(tags)

Note that mvfind uses regex so you may get some odd results if your tags have special characters in them

Re: how to search in array inside a foreach loop

yuanliu — Thu, 16 Jan 2025 04:41:12 GMT

I see you want to determine full paths of the value input list. You have a second requirement that the input be a JSON array, ["Tag3", "Tag4"], and a third that the code needs to run in 8.0, which precludes JSON functions introduced in 8.1. Note each of the path{} array has multiple values. Without help of JSON functions, you need to handle that first.

The most common way to do this is with mvexpand. (The input array also needs this.)

| makeresults | eval _raw = "{ \"Info\": { \"Apps\": { \"ReportingServices\": { \"ReportTags\": [ \"Tag1\" ], \"UserTags\": [ \"Tag2\", \"Tag3\" ] }, \"MessageQueue\": { \"ReportTags\": [ \"Tag1\", \"Tag4\" ], \"UserTags\": [ \"Tag3\", \"Tag4\", \"Tag5\" ] }, \"Frontend\": { \"ClientTags\": [ \"Tag12\", \"Tag47\" ] } } } }" | spath ``` data emulation above ``` | eval Tags = "[\"Tag3\", \"Tag4\"]" | foreach *Tags{} [mvexpand <<FIELD>>] | spath input=Tags | mvexpand {} | foreach *Tags{} [eval tags=mvappend(tags, if(lower('<<FIELD>>') = lower('{}'), "<<FIELD>>", null()))] | dedup tags | stats values(tags)

If your dataset is large, mvexpand has some limitations.