Splunk Search

Configuring field extractions for multivalue nested JSON events

sharad06
Explorer

Hi experts,

I am working with nested JSON events which look as follows:

{   [-] 
     compliance:  <compliance_state>    
     ctupdate:  hostinfo    
     host_properties:   {   [-] 
         hwi_computer:  [   [+] 
        ]   
         hwi_disk:  [   [-] 
        {   [-] 
             counteract_tag:     health_data    
             since:  1519929479 
             value:  Device ID: C:;Drive Type: Local Disk;Free Space (Megabytes): 93159.7578125;Size (Megabytes): 122776.99609375;Name: C:;Description: Local Fixed Disk;Media Type: Fixed hard disk media;File System: NTFS    
        }   
        {   [-] 
             counteract_tag:     health_data    
             since:  1519929479 
             value:  Device ID: A:;Drive Type: Removable Disk;Name: A:;Description: 3 1/2 Inch Floppy Drive;Media Type: 3 1/2-Inch Floppy Disk - 720 KB - 512 bytes/sector; 
        }   
        {   [-] 
             counteract_tag:     health_data    
             since:  1519929479 
             value:  Device ID: D:;Drive Type: Compact Disc;Name: D:;Description: CD-ROM Disc;Media Type: Removable media other than floppy;    
        }   
        ]   
         hwi_network_adapters:  [   [+] 
        ]   
         hwi_physical_memory:   [   [+] 
        ]   
         hwi_processor: [   [+] 
        ]   
    }   
     ip:     <ipv4_address> 
     ipv6:   <ipv6_address> 
     mac:    <mac_address>  
     nbtdomain:  <domain>   
     user:   <user>
}

The above event contains a host_property named 'hwi_disk' which contains another JSON array carrying disk details of an endpoint under the key named 'value'. This 'value' contains different sub-keys separated by ':' and ';'. So far, I've been writing search queries using regexes to create my dashboards. The search queries look like:

`get_index` `get_sourcetypes` `ct_hostinfo` hwi_disk
| rename host_properties.hwi_disk{}.value as hwi_disk_val
| mvexpand hwi_disk_val
| rex field=hwi_disk_val "Device ID:\s*(?<DeviceID>[^;^$]*).*Free Space \(Megabytes\):\s*(?<FreeSpace>[^;^$]*).*Size \(Megabytes\):\s*(?<Size>[^;^$]*).*"
| stats latest(FreeSpace) as "Disk Space Available (MB)" latest(Size) as "Capacity (MB)" by ip DeviceID
| sort limit=10 -"Disk Space Available (MB)"
| rename ip as IP DeviceID as "Device ID"

I would like to know if I can write field extractions for such a multi-value event using conf files? Some pointers on how such extractions can written would be great help!

Thanks.

0 Karma

valiquet
Contributor

You can use KV_MODE=JSON on SH if your JSON syntax is valid.

0 Karma

p_gurav
Champion

Hi,

Can you try something like this in search:

  | extract pairdelim="\r\n" kvdelim=":"

Let me know what it gives.

0 Karma

sharad06
Explorer

Hi p_gaurav,

Thanks for your reply. I used the following query:

`get_index` `get_sourcetypes` `ct_hostinfo` hwi_network_adapters
| rename host_properties.hwi_network_adapters{}.value as hwi_network_adapters_val
| mvexpand hwi_network_adapters_val
| extract pairdelim="\r\n" kvdelim=":"

But it doesn't extract any of the nested values. I can see why as well because I want to run this 'extract command' on 'hwi_network_adapters_val' field, but to my knowledge, I can't specify a field parameter with 'extract' command (like I can do with the 'rex' command).

Finally, the greater challenge is to figure out how will this field extraction config translate to props.conf and transforms.conf stanzas.

Thanks for your help.

0 Karma

sharad06
Explorer

KV_MODE=JSON can not extract fields for nested JSON events.

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Get More Out of Your Security Practice With a SIEM

Get More Out of Your Security Practice With a SIEMWednesday, July 31, 2024  |  11AM PT / 2PM ETREGISTER ...