topic Re: How do you parse nested Amazon Web Services fields? in Getting Data In

How do you parse nested Amazon Web Services fields?

MABurberry — Mon, 14 Jan 2019 09:08:32 GMT

Hi All,

I am having some troubles parsing nested AWS fields.

The data that I have looks like this:

     rules: [   [-] 
        {   [-] 
         from_port:  80 
         grants:    [   [-] 
            {   [-] 
             cidr_ip:    10.51.4.20/31      
             group_id:   null   
             name:   null   
             owner_id:   null   
            }   
            {   [-] 
             cidr_ip:    10.51.4.8/31       
             group_id:   null   
             name:   null   
             owner_id:   null   
            }   
            {   [-] 
             cidr_ip:    10.51.4.2/31   
             group_id:   null   
             name:   null   
             owner_id:   null   
            }   
        ]   
         groups:    
         ipRanges:   

         ip_protocol:    tcp    
         to_port:    80 
        }   
        {   [-] 
         from_port:  0  
         grants:    [   [-] 
            {   [-] 
             cidr_ip:    10.0.1.9/21    
             group_id:   null   
             name:   null   
             owner_id:   null   
            }   
        ]   
         groups:    
         ipRanges:   

         ip_protocol:    tcp    
         to_port:    65535  
        }   
        {   [-] 
         from_port:  7002   
         grants:    [   [-] 
            {   [-] 
             cidr_ip:    10.0.1.7/21    
             group_id:   null   
             name:   null   
             owner_id:   null   
            }   
            {   [-] 
             cidr_ip:    10.0.1.5/21    
             group_id:   null   
             name:   null   
             owner_id:   null   
            }   
            {   [-] 
             cidr_ip:    10.0.1.2/21    
             group_id:   null   
             name:   null   
             owner_id:   null   
            }   
        ]   
         groups:    
         ipRanges:   

         ip_protocol:    tcp    
         to_port:    7002   
        }

I want to be able to parse these fields so they show up like:

IP Address      FROM_PORT TO_PORT
10.51.4.20/31 80                   80
10.51.4.8/31   80                   80
10.51.4.2/31   80                   80
10.0.1.9/21     0                     65535
10.0.1.7/21 7002               7002

I've tried MVZip then MVExpand but I cannot seem to get it working correctly. Does anyone have any ways to solve this please?

Thanks

Re: How do you parse nested Amazon Web Services fields?

493669 — Mon, 14 Jan 2019 09:40:28 GMT

Hi @MABurberry,

can you try |spath command like below

index=<YourIndexname>|spath

https://docs.splunk.com/Documentation/Splunk/7.2.3/SearchReference/Spath

Re: How do you parse nested Amazon Web Services fields?

MABurberry — Mon, 14 Jan 2019 10:12:48 GMT

Thanks @493669

I was able to extract the correct data using spath.

My search was:
index=test
| spath path=rules{} output=rules
| mvexpand rules
| rename rules as _raw
| spath

Re: How do you parse nested Amazon Web Services fields?

dkeck — Mon, 14 Jan 2019 10:17:50 GMT

Hi,

you need a bit more than just spath

in this example you can see how to use spath in the way you wanted your example

| makeresults | eval _raw = "{\"rules\": [{\"from\": 1, \"grant\":[{\"ip\": 12}, {\"ip\": 34}]}, {\"from\": 2, \"grant\":[{\"ip\": 56}, {\"ip\": 78}]}]}" | spath rules{} output=rules | mvexpand rules | spath input=rules

use just the part after |spath and change the names.

David