Dealing with badly structured json events

[david_rose]

I am pulling in some json events that are poorly structured (at least for my needs). Specifically, I need to be able to tie the VULNS data to its ip address, which is above it. But since they are not nested, splunk is not able to make the connection. Is there any way to tie the IP address to the data below it or readjust the nesting of the event in a search?

Example data below

vulnerable_hosts: [ [-] 
         { [-] 
            ip_address:  xxx.xxx.xxx.xxx 
            vulns: [ [-] 
             { [-] 
                risk_level:  High 
                service_port:  443 
                service_protocol:  TCP 
                title:  SSL/TLS server supports RC4 ciphers 
             } 
             { [-] 
                risk_level:  Medium 
                service_port:  80 
                service_protocol:  TCP 
                title:  TCP timestamp requests enabled 
             } 
             { [-] 
                risk_level:  Medium 
                service_port:  443 
                service_protocol:  TCP 
                title:  SSL certificate is signed with weak hash function: SHA1 
             } 
           ] 
         } 
         { [-] 
            ip_address:  yyy.yyy.yyy.yyy 
            vulns: [ [-] 
             { [-] 
                risk_level:  Urgent 
                service_port:  161 
                service_protocol:  UDP 
                title:  SNMP is enabled and may be vulnerable 
             } 
             { [-] 
                risk_level:  Low 
                service_port:  0 
                service_protocol:  ICMP 
                title:  ICMP timestamp requests enabled 
             } 
           ] 

[martin_mueller · Accepted solution]

You need a two-step approach:

... | spath vulnerable_hosts{} output=vulnerable_hosts | mvexpand vulnerable_hosts | spath input=vulnerable_hosts

First you extract the objects inside the vulnerable_hosts array, then you turn them into individual events, finally you extract the content of those objects.

[david_rose]

Thanks Martin. But with this method, how can I connect the vulnerability to the the correct ip ?

[david_rose]

Ignore that, it was there, just not presented in the events. You rock!