Splunk Search

Configuring field extractions for multivalue nested JSON events

sharad06
Explorer

Hi experts,

I am working with nested JSON events which look as follows:

{   [-] 
     compliance:  <compliance_state>    
     ctupdate:  hostinfo    
     host_properties:   {   [-] 
         hwi_computer:  [   [+] 
        ]   
         hwi_disk:  [   [-] 
        {   [-] 
             counteract_tag:     health_data    
             since:  1519929479 
             value:  Device ID: C:;Drive Type: Local Disk;Free Space (Megabytes): 93159.7578125;Size (Megabytes): 122776.99609375;Name: C:;Description: Local Fixed Disk;Media Type: Fixed hard disk media;File System: NTFS    
        }   
        {   [-] 
             counteract_tag:     health_data    
             since:  1519929479 
             value:  Device ID: A:;Drive Type: Removable Disk;Name: A:;Description: 3 1/2 Inch Floppy Drive;Media Type: 3 1/2-Inch Floppy Disk - 720 KB - 512 bytes/sector; 
        }   
        {   [-] 
             counteract_tag:     health_data    
             since:  1519929479 
             value:  Device ID: D:;Drive Type: Compact Disc;Name: D:;Description: CD-ROM Disc;Media Type: Removable media other than floppy;    
        }   
        ]   
         hwi_network_adapters:  [   [+] 
        ]   
         hwi_physical_memory:   [   [+] 
        ]   
         hwi_processor: [   [+] 
        ]   
    }   
     ip:     <ipv4_address> 
     ipv6:   <ipv6_address> 
     mac:    <mac_address>  
     nbtdomain:  <domain>   
     user:   <user>
}

The above event contains a host_property named 'hwi_disk' which contains another JSON array carrying disk details of an endpoint under the key named 'value'. This 'value' contains different sub-keys separated by ':' and ';'. So far, I've been writing search queries using regexes to create my dashboards. The search queries look like:

`get_index` `get_sourcetypes` `ct_hostinfo` hwi_disk
| rename host_properties.hwi_disk{}.value as hwi_disk_val
| mvexpand hwi_disk_val
| rex field=hwi_disk_val "Device ID:\s*(?<DeviceID>[^;^$]*).*Free Space \(Megabytes\):\s*(?<FreeSpace>[^;^$]*).*Size \(Megabytes\):\s*(?<Size>[^;^$]*).*"
| stats latest(FreeSpace) as "Disk Space Available (MB)" latest(Size) as "Capacity (MB)" by ip DeviceID
| sort limit=10 -"Disk Space Available (MB)"
| rename ip as IP DeviceID as "Device ID"

I would like to know if I can write field extractions for such a multi-value event using conf files? Some pointers on how such extractions can written would be great help!

Thanks.

0 Karma

valiquet
Contributor

You can use KV_MODE=JSON on SH if your JSON syntax is valid.

0 Karma

p_gurav
Champion

Hi,

Can you try something like this in search:

  | extract pairdelim="\r\n" kvdelim=":"

Let me know what it gives.

0 Karma

sharad06
Explorer

Hi p_gaurav,

Thanks for your reply. I used the following query:

`get_index` `get_sourcetypes` `ct_hostinfo` hwi_network_adapters
| rename host_properties.hwi_network_adapters{}.value as hwi_network_adapters_val
| mvexpand hwi_network_adapters_val
| extract pairdelim="\r\n" kvdelim=":"

But it doesn't extract any of the nested values. I can see why as well because I want to run this 'extract command' on 'hwi_network_adapters_val' field, but to my knowledge, I can't specify a field parameter with 'extract' command (like I can do with the 'rex' command).

Finally, the greater challenge is to figure out how will this field extraction config translate to props.conf and transforms.conf stanzas.

Thanks for your help.

0 Karma

sharad06
Explorer

KV_MODE=JSON can not extract fields for nested JSON events.

0 Karma
Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...