Applying an inputlookup across a list of values

[bpenny]

We have a use case where some JSON being ingested into Splunk contains a list of values like this:

        "message_set": [
            {
                "type": 9
            },
            {
                "type": 22
            },
            {
                "type": 15
            },
...
        ],

That list has an arbitrary length, so it could contain anywhere from one up to around 30 "type" values. Splunk is parsing the JSON just fine, so these fields can be referenced as "message_info.message_set{}.type" in searches.

I'd like to set up an inputlookup that maps these numerical values to more descriptive text. Is there a way to apply an inputlookup across an entire list of arbitrary size like this, or would I need to explicitly add an inputlookup definition for each individual index in the list? I'd ultimately like to add these as LOOKUP settings in the sourcetype for this data so that they're automatically applied for all searches.

[livehybrid]

Hi @bpenny 

You should be able to do a simple lookup for this, something like this:

| lookup typesEnrich.csv type AS msg.message_set{}.type OUTPUT typeDescription

To demonstrate this I've created a sample lookup file:

| makeresults count=1
| eval type=1, typeDescription="Type A"
| append [ | makeresults count=1 | eval type=2, typeDescription="Type B" ]
| append [ | makeresults count=1 | eval type=3, typeDescription="Type C" ]
| append [ | makeresults count=1 | eval type=4, typeDescription="Type D" ]
| append [ | makeresults count=1 | eval type=5, typeDescription="Type E" ]
| append [ | makeresults count=1 | eval type=6, typeDescription="Type F" ]
| table type typeDescription
| outputlookup typesEnrich.csv

Then using some sample data we can emulate your use-case (hopefully!)

| makeresults
| eval json_data = "{\"msg\":{\"message_set\": [{\"type\": 1}, {\"type\": 2}, {\"type\": 4}]}}"
| eval _raw=json_extract(json_data,"")
| table _raw
| spath input=_raw
| lookup typesEnrich.csv type AS msg.message_set{}.type OUTPUT typeDescription

Which gives the following:

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

[bpenny]

Thanks, @livehybrid, this is very close to what I need. What I ultimately want though, is to make these automatic lookups. We actually have about ten different ones that we need to apply to this particular sourcetype. I just can't seem to figure out how to add something like msg.message_set{}.type to an automatic lookup and have it work.

[livehybrid · Accepted solution]

Hi @bpenny 

If you're looking to do it as an automatic lookup then you should be able to use the following, configured from Settings -> Lookups -> Automatic Lookups.

Or as a props.conf:

[yourSourceType]
LOOKUP-lookup1 = yourLookupName type AS "msg.message_set{}.type" OUTPUTNEW typeDescription AS typeDescription

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

[richgalloway]

Check out the lookup function.  It should do what you want and put the results in a separate JSON array.