Getting all IP addresses from guests in VMware vCenter

[bensec01]

Hey folks,

I am using a VMware DCN (data collection node) to index all of my performance, event, and inventory data from our VMware environment. This is a standalone Splunk Enterprise instance running 6.6.2, not a virtual appliance.

I am working on a set of dashboards to lookup VMs, hypervisors, etc, and I'm seeing some weirdness in the data. I'm hoping someone can help me out, because my brain doesn't grok JSON and I have a helluva time with multivalue fields.

Here is the search I am running that gives some weirdness. I am specifically looking at

tag=virtualmachine tag=inventory tag=virtualization vm_name="*" 
| stats delim="," latest(vm_name) as vm_name, latest(storage_capacity) as storage_capacity, latest(mem_capacity) as mem_capacity, latest(processor_socket_count) as processor_socket_count, latest(cpu_cores) as cpu_cores, latest(logical_cpu_count) as logical_cpu_count, latest(power_state) as power_state, latest(vm_os) as vm_os, values(ip) as ip, values(datastore) as datastore, values(datastore_volume_path) as datastore_volume_path, latest(cluster_id) as cluster_id, latest(cluster_name) as cluster_name, latest(hypervisor_name) as hypervisor_name, by vm_id 
| eval storage_capacity=round(storage_capacity/1024/1024/1024) 
| eval mem_capacity=round(mem_capacity/1024/1024/1024) 
| eval hypervisor_name=upper(mvindex(split(hypervisor_name,"."),0))
| rename vm_id as "VM ID", vm_name as "VM Name", storage_capacity as "Storage Capacity (in GB)", mem_capacity as "Memory Capacity (in GB)", processor_socket_count as "CPU Socket Count", cpu_cores as "CPU Cores", logical_cpu_count as "Logical CPU Count", power_state as "Power State", vm_os as "VM Operating System", ip as "IP Address(es)", datastore as "Datastore(s)", datastore_volume_path as "Datastore Volume Path", hypervisor_name as "Found on Hypervisor", cluster_id as "Cluster ID", cluster_name as "Cluster Name"

Now, this appears to work, but I only get the last IP address for guests that have multiple IPs assigned. If I look at the event itself, I find the list of IP addresses for the guest in:

{   [-] 
   changeSet: { [-] 
     guest: {   [-] 
       disk: {  [+] 
       }    
       guestFamily: linuxGuest  
       guestFullName: Red Hat Enterprise Linux 7 (64-bit)   
       guestId: rhel7_64Guest   
       guestOperationsReady: True   
       guestState: running  
       hostName: hostname   
       ipAddress: just_one_of_the_ip_addresses
       ipStack: {   [+] 
       }    
       net: {   [-] 
         GuestNicInfo: [    [-] 
           {    [-] 
             connected: True    
             deviceConfigId: 4000   
             ipAddress: [   [-] 
               ip_address_1 
               ip_address_2 
               ip_address_3 
             ]  
             ipConfig: {    [+] 
             }  
             macAddress: 00:aa:bb:cc:dd:ee  
             network: myVLAN    
           }    
         ]  
       }    
...

My search is giving me the just_one_of_the_ip_addresses value, while I see the correct list of them in ip_address_1, _2, _3. But I have no idea how to access them and display the list of them.

Can someone help my brain wrap around the JSON + multivalue field thing?

Thank you!

Chris

[woodcock]

The command that you need is spath:
https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Spath