how to search in array inside a foreach loop

[mrsampson]

This is an example of the structure of my data and the query I am currently using. I have tried around 10 different solutions based on various examples from stackoverflow.com and  community.splunk.com. But I have not figured out how to change this query such that eval Tag = "Tag1" can become an array eval Tags = ["Tag1", "Tag4"] and I will get entries for all tags that exist in the array. Could someone guide me in the right direction?

| makeresults
| eval _raw = "{
    \"Info\": {
        \"Apps\": {
            \"ReportingServices\": {
                \"ReportTags\": [
                    \"Tag1\"
                ],
                \"UserTags\": [
                    \"Tag2\",
                    \"Tag3\"
                ]
            },
            \"MessageQueue\": {
                \"ReportTags\": [
                    \"Tag1\",
                    \"Tag4\"
                ],
                \"UserTags\": [
                    \"Tag3\",
                    \"Tag4\",
                    \"Tag5\"
                ]
            },
            \"Frontend\": {
                \"ClientTags\": [
                    \"Tag12\",
                    \"Tag47\"
                ]
            }
        }
    }
}"
| eval Tag = "Tag1"
| spath
| foreach *ReportTags{}
    [| eval tags=mvappend(tags, if(lower('<<FIELD>>') = lower(Tag), "<<FIELD>>", null()))]
| dedup tags
| stats values(tags)

[mrsampson]

Does anyone know how to do this on Splunk v8.0.5?

[ITWhisperer · Accepted solution]

Try something like this

| eval Tag = split("Tag3,Tag4",",")
| mvexpand Tag
| spath
| foreach *Tags{}
    [| eval tags=if(mvfind(lower('<<FIELD>>'), "^".lower(Tag)."$") >= 0,mvappend(tags, "<<FIELD>>"), tags)]
| stats values(tags)

Note that mvfind uses regex so you may get some odd results if your tags have special characters in them

[yuanliu]

Do you mean this?

| makeresults
| eval _raw = "{
    \"Info\": {
        \"Apps\": {
            \"ReportingServices\": {
                \"ReportTags\": [
                    \"Tag1\"
                ],
                \"UserTags\": [
                    \"Tag2\",
                    \"Tag3\"
                ]
            },
            \"MessageQueue\": {
                \"ReportTags\": [
                    \"Tag1\",
                    \"Tag4\"
                ],
                \"UserTags\": [
                    \"Tag3\",
                    \"Tag4\",
                    \"Tag5\"
                ]
            },
            \"Frontend\": {
                \"ClientTags\": [
                    \"Tag12\",
                    \"Tag47\"
                ]
            }
        }
    }
}"
| eval Tag = "Tag1"
| spath
| foreach *ReportTags{}
    [| eval tags=mvappend(tags, if(lower('<<FIELD>>') = lower(Tag), '<<FIELD>>', null()))]
| dedup tags
| stats values(tags)

This gives

values(tags)

Tag1 Tag4

Note when you use double quote on the right-hand side of an eval expression, in quoted entity is used as literal, therefore your original search gives

values(tags)

Info.Apps.MessageQueue.ReportTags{} Info.Apps.ReportingServices.ReportTags{}

[mrsampson]

not really, the main point in here is that my input to this query, instead of a simple value would be an array. e.g.
current input format: 

| eval Tag = "Tag1"

desired input format: 

| eval Tags = ["Tag3", "Tag4"]

[ITWhisperer]

What does your expected output look like?

[mrsampson]

for this

| eval Tags = ["Tag3", "Tag4]
| spath
| foreach *Tags{}
    [| eval tags=mvappend(tags, if(lower('<<FIELD>>') = lower(Tag), "<<FIELD>>", null()))]
| dedup tags
| stats values(tags)

I would like to get

Info.Apps.MessageQueue.ReportTags{}
Info.Apps.ReportingServices.ReportTags{}
Info.Apps.MessageQueue.UserTags{}​

[yuanliu]

I see you want to determine full paths of the value input list.  You have a second requirement that the input be a JSON array,  ["Tag3", "Tag4"], and a third that the code needs to run in 8.0, which precludes JSON functions introduced in 8.1.  Note each of the path{} array has multiple values.  Without help of JSON functions, you need to handle that first.

The most common way to do this is with mvexpand. (The input array also needs this.)

| makeresults
| eval _raw = "{
    \"Info\": {
        \"Apps\": {
            \"ReportingServices\": {
                \"ReportTags\": [
                    \"Tag1\"
                ],
                \"UserTags\": [
                    \"Tag2\",
                    \"Tag3\"
                ]
            },
            \"MessageQueue\": {
                \"ReportTags\": [
                    \"Tag1\",
                    \"Tag4\"
                ],
                \"UserTags\": [
                    \"Tag3\",
                    \"Tag4\",
                    \"Tag5\"
                ]
            },
            \"Frontend\": {
                \"ClientTags\": [
                    \"Tag12\",
                    \"Tag47\"
                ]
            }
        }
    }
}"
| spath
``` data emulation above ```
| eval Tags = "[\"Tag3\", \"Tag4\"]"
| foreach *Tags{}
    [mvexpand <<FIELD>>]
| spath input=Tags
| mvexpand {}
| foreach *Tags{}
    [eval tags=mvappend(tags, if(lower('<<FIELD>>') = lower('{}'), "<<FIELD>>", null()))]
| dedup tags
| stats values(tags)

If your dataset is large, mvexpand has some limitations.

[ITWhisperer]

Splunk's version of arrays is multivalue field, so if you change you input to a multivalue field, you could do something like this

| eval Tag = split(lower("Tag3,Tag4"),",")
| spath
| foreach *Tags{}
    [| eval field="<<FIELD>>"
    | foreach <<FIELD>> mode=multivalue
        [| eval tags=if(isnull(tags),if(mvfind(Tag,lower('<<ITEM>>')) >= 0, field, null()),mvappend(tags, if(mvfind(Tag,lower('<<ITEM>>')) >= 0, field, null())))]
    ]
| stats values(tags)

[mrsampson]

---

@ITWhisperer wrote:

Splunk's version of arrays is multivalue field, so if you change you input to a multivalue field, you could do something like this

| eval Tag = split(lower("Tag3,Tag4"),",")
| spath
| foreach *Tags{}
    [| eval field="<<FIELD>>"
    | foreach <<FIELD>> mode=multivalue
        [| eval tags=if(isnull(tags),if(mvfind(Tag,lower('<<ITEM>>')) >= 0, field, null()),mvappend(tags, if(mvfind(Tag,lower('<<ITEM>>')) >= 0, field, null())))]
    ]
| stats values(tags)

---

Thank you for your response and the example, currently it is returnin 0 results for me. Could it have something to do with my Splunk version? I a

[mrsampson]

m using 8.0.5

[PickleRick]

Yes, foreach mode=multivalue appeared in 9.0.0.